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Foreword 



This volume contains the papers presented at the 7th International Workshop on 
Approximation Algorithms for Combinatorial Optimization Problems (APPROX 
2004) and the 8th International Workshop on Randomization and Computa- 
tion (RANDOM 2004), which took place concurrently at Harvard University, 
Cambridge, on August 22-24, 2004. APPROX focuses on algorithmic and com- 
plexity issues surrounding the development of efficient approximate solutions to 
computationally hard problems, and this year’s workshop was the seventh in 
the series after Aalborg (1998), Berkeley (1999), Saarbriicken (2000), Berkeley 
(2001), Rome (2002), and Princeton (2003). RANDOM is concerned with appli- 
cations of randomness to computational and combinatorial problems, and this 
year’s workshop was the eighth in the series following Bologna (1997), Barcelona 
(1998), Berkeley (1999), Geneva (2000), Berkeley (2001), Harvard (2002), and 
Princeton (2003). 

Topics of interest for APPROX and RANDOM are: design and analysis of 
approximation algorithms, inapproximability results, approximation classes, on- 
line problems, small space and data streaming algorithms, sub-linear time algo- 
rithms, embeddings and metric space methods in approximation, math progam- 
ming in approximation algorithms, coloring and partitioning, cuts and connec- 
tivity, geometric problems, network design and routing, packing and covering, 
scheduling, game theory, design and analysis of randomized algorithms, ran- 
domized complexity theory, pseudorandomness and derandomization, random 
combinatorial structures, random walks/Markov chains, expander graphs and 
randomness extractors, probabilistic proof systems, random projections and em- 
beddings, error-correcting codes, average-case analysis, property testing, compu- 
tational learning theory, and other applications of approximation and random- 
ness. 

The volume contains 19-1-18 contributed papers, selected by the two program 
committees from 54-1-33 submissions received in response to the call for papers. 

We would like to thank all of the authors who submitted papers, the members 
of the program committees 

APPROX 2004 

Chandra Chekuri, Bell Laboratories 
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Sanjeev Khanna, U. of Pennsylvania (Chair) 

Rajmohan Rajaraman, Northeastern U. 

Tim Roughgarden, UC Berkeley 
Baruch Schieber, IBM T.J. Watson 
Martin Skutella, Max Planck Institute 
Dan Spielman, MIT 
Luca Trevisan, UC Berkeley 
Mihalis Yannakakis, Columbia U. 

Neal Young, UC Riverside 
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Adam Kalai, Toyota Technological Institute 
Satish Rao, UC Berkeley 

Dana Ron, Tel Aviv U. and Radcliffe Institute, Harvard (Chair) 
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Neal Young, UC Riverside 

and the external subreferees Udo Adamy, Dorit Aharonov, Ernst Althaus, Javed 
Aslam, Nikhil Bansal, Therese Beidl, Petra Berenbrink, Joan Boyar, Gruia 
Calinescu, Kamalika Chaudhuri, Bernard Chazelle, Jiangzhuo Chen, Marek 
Chrobak, Colin Cooper, Erik Demaine, Nikhil Devanur, Thomas Erlebach, 
Martin Farach-Colton, Sandor Fekete, Tom Friedetzky, Rajiv Gandhi, Mordecai 
Colin, Oded Goldreich, Alex Hall, Refael Hassin, Stefan Hougardy, Piotr Indyk, 
Wojtek Jawor, Mark Jerrum, Lujun Jia, Valentine Kabanets, Marek Karpinski, 
Tali Kaufman, Julia Kempe, Claire Kenyon, Tracy Kimbrel, Robert Kleinberg, 
Adam Klivans, Jochen Konemann, Stavros Kolliopoulos, Madhular Korupolu, 
Robert Krauthgamer, Kofi Laing, Matt Levine, Moshe Lewenstein, Guolong 
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Abstract. We study a problem motivated by a scheme for supporting fast restora- 
tion in MPLS and optical networks. In this local restoration scheme detour paths 
are set-up a priori and network resources are pre-reserved exclusively for car- 
rying rerouted traffic under network failures, (i.e. they do not carry any traffic 
under normal working conditions). The detours are such that failed links can be 
bypassed locally from the first node that is upstream from the failures. This local 
bypass activation from the first detection point for failures along with the dedica- 
tion of network resources for handling failures permits very fast recovery times, 
a critical requirement for these networks. By allowing sharing of the dedicated 
resources among different detours the local restoration scheme results in efficient 
utilization of the pre-reserved network capacity. 

In this paper we are interested in the problem of dedicating the least amount of 
the currently available network capacity for protection, while guaranteeing fast 
restoration to the existing traffic along with any traffic that may be admitted in 
the future. We show that the problem is NP-hard, and give a 2-approximation 
algorithm for the problem. We also show that the integrality gap of a natural re- 
laxation of our problem is i?(n), thus establishing that any LP-based approach 
using this relaxation cannot yield a better approximation algorithm for our prob- 
lem. 



1 Introduction 

Dynamic provisioning of bandwidth guaranteed paths with fast restoration capability is 
an important network service feature for the emerging Multi-Protocol Label Switched 
(MPLS) networks [7] and optical mesh networks [18]. The fast restoration capabilities 
are required in order to provide the needed reliability for services such as packetized 
voice, critical VPN traffic, etc. Traditionally ring based SONET [11] networks have 
offered 50ms restoration to bandwidth guaranteed services, using pre-reserved spare 
protection capacity and pre-planned protection paths. Pre-planning protection in rings 
has been especially attractive, because of the availability of exactly one backup path 
between any two nodes, leading to very simple and fast automatic protection switching 
mechanisms. However in ring based SONET networks these advantages come at the 
cost of reserving at least half the total capacity for protection. 

A local restoration scheme [12], [16], [20] is proposed to provide fast restoration 
in mesh based MPLS and optical networks. In this scheme which is also referred to as 

K. Jansen et al. (Eds.): APPROX and RANDOM 2004, LNCS 3122, pp. 1-12, 2004. 

© Springer- Verlag Berlin Heidelberg 2004 
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link restoration the traffic on each link e of the network is protected by a detour path 
that does not include link e. Upon failure of any link e, any traffic on e is switched to its 
detour path. Thus, link restoration provides a local mechanism to route around a failure. 
In this restoration scheme the restoration capacity of the pre-setup detours is not used 
under normal no-failure conditions (except possibly by low priority preemptible traffic). 
Local restoration when used in conjunction with advanced reservation of the restoration 
capacities and pre-setup detours results in low restoration latency. Pre-provisioned link 
restoration also results in operational simplicity since the detours have to be only provi- 
sioned once for a given network topology and since the online connection routing can 
now be done oblivious to the reliability requirements, using only the resources that are 
not reserved for restoration. 

An important consideration for any fast restoration scheme is to minimize the net- 
work resources dedicated for restoration and hence to maximize the proportion of net- 
work resources available for carrying traffic under normal working conditions. In gen- 
eral, link restoration provides guaranteed protection against only single link failures 
hence the reserved restoration capacity may be shared among the different pre-setup 
detours (since at most one detour may carry restored traffic at any given time). Thus 
pre-provisioned link restoration scheme offers the promise of fast restoration recovery 
for just a small fraction of the total capacity reserved for restoration, due to the high 
degree of restoration capacity sharing that is possible in mesh networks. 

In many situations one would like to support fast restoration on existing networks 
without disturbing the existing traffic, meaning that the restoration scheme can only 
use up to the current available network capacity (link capacity minus existing traffic 
capacity) for protecting the existing traffic and any new traffic. Note that existing traffic 
makes the problem harder. A simpler polynomial-time 2-approximation algorithm in 
the absence of existing traffic was presented in [1] (see Related Work). 

In this paper we are interested in the optimization problem of dedicating the least 
amount of the currently available network capacity for protection, while guaranteeing 
fast restoration to the existing traffic along with any traffic that may be admitted in the 
future, for the pre-provisioned link restoration scheme. Specifically we are interested in 
partitioning the available link capacities into working and protection, such that the latter 
is dedicated for restoration and the former is available to carry any current or new traffic, 
with the objective of guaranteeing link restoration for minimal total protection capacity. 
Note that in a network with a static topology this problem may need be solved only 
once, since the solution remains feasible even as the admitted traffic pattern changes. 
However, the solution may not stay optimal over time as the admitted traffic pattern 
changes, and may be recomputed occasionally to ensure efficient utilization of network 
resources. Also changes in network topology (which are common but not frequent) 
may require recomputing the solution, since the old solution may not even guarantee 
link restoration. 

1.1 Problem Definition 

Given a undirected network G = (V,E), with link capacities Ue and existing traffic We 
on link e G E the problem is to partition the capacities on link e into a working capacity 
We and a protection capacity pe (s.t. We + Pe = Ue) such that 



Designing Networks with Existing Traffic to Support Fast Restoration 



3 



- The total protection capacity rninimized. 

- For every link e = {u, v), in the network G — e obtained by removing link e from 
G, there exists a path Pg (detour for link e) between nodes u and v, such that every 
link e' on Pg satisfies > Wg. 

- The working capacity Wg is at least the amount of the existing traffic Wg . 

- In case such a partition is not feasible on G, output an empty solution. 

In other words, on link e, pg capacity is reserved for carrying restored traffic during 
failures and at most Wg {wg > Wg) traffic is carried during normal working conditions. 
Thus on the failure of link e, at most Wg traffic on it is rerouted over the pre-setup detour 
path Pg using only the reserved capacities on the links on Pg. 

Note that given the pg and Wg values for all links e in any feasible solution, the 
detour paths Pg can be easily computed. Hence we do not include the computation of 
the detour paths Pg in the statement of the problem. 

1.2 Our Contribution 

We show that given an instance of the problem, it can be determined in polynomial time, 
using a fast and efficient algorithm, if the problem has a feasible solution. However, we 
show that computing an optimal solution for feasible instances is NP-hard. Moreover, 
we present a simple and efficient algorithm that computes a solution to the given feasible 
instance in which the total protection capacity reserved is guaranteed to be within two 
times the protection capacity reserved in any optimal solution. We also show that the 
integrality gap of a natural relaxation of our problem is I7(n), thus establishing that any 
LP-based approach using this relaxation cannot yield a better approximation algorithm 
for our problem. 

2 Related Work 

The main approaches for supporting pre-provisioned link restoration scheme in mesh 
networks are based on identifying ring structures. Once the set of rings are identified 
then pre-planned restoration schemes as in SONET [11] are employed. In some of these 
approaches the network is designed in term of rings [17] or by partially using rings [9]. 
Thus, these schemes are only applicable to constrained topologies. In some other of 
these approaches each link is covered by a cycle leading to a cycle cover for the net- 
work [9]. Each of these cycles is then provisioned with enough protection capacity to 
cover the links that belong to it. On the failure of the link the working traffic is rerouted 
over the protection capacities in the surviving links of the covering cycle. There are 
two drawbacks of these approaches: first the amount of pre-provisioned protection ca- 
pacity can be significant and second it is hard to find the smallest cycle cover of a 
given network [19]. An improvement to these schemes is those based on the notion 
of p-cycle [10]. Here the main idea is that a cycle can be used to protect not just the 
links on the cycle but also the chords (spokes) of the cycle, thus showing that far fewer 
cycles (than in a cycle cover) may be sufficient for providing full protection. An algo- 
rithm to minimize the total spare capacity, based on solving an integer program over 
all possible cycles is given in [10]. To the best of our knowledge no fast approxima- 
tion algorithms for this problem are known. An alternative to cycle covers, intended to 
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overcome the difficulty of finding good covers, is to cover every link in a network with 
exactly two cycles [8]. A set of cycles that meets this requirement is called a double cy- 
cle cover [13]. For planar graphs, double cycle covers can be found in polynomial-time. 
For non-planar graphs, it is conjectured that double cycle covers exist, and they are typ- 
ically found quickly in practice. However, even for double cycle cover based protection 
schemes, the required pre-provisioned protection capacity can be significant. 

Non-ring based approaches to link restoration on mesh networks include general- 
ized loop-back [14], [15], where the main idea is to select a digraph, called the primary, 
such that the conjugate digraph, called the secondary, can be used to carry the switched 
traffic for any link failure in the primary. Chekuri et al. [5] consider the problem of 
adding protection capacity to the links of a given network (primary) carrying working 
traffic, at minimum cost, so that the resulting network is capable of supporting link pro- 
tection for a given set of links, where the protection is provided to the working traffic 
on the primary network. In their model no limit is imposed on the total capacities of the 
links, and they provide a 4-approximation algorithm when all links in the original pri- 
mary network have uniform bandwidth (carrying the same amount of working traffic) 
and they provide a 10.87-approximation algorithm for the general case. In addition they 
also provide a 0(log n) -approximation algorithm for the problem of jointly designing 
the primary and protected networks, given a demand matrix for the working traffic. 

Our previous work [1] considers the problem under the assumption that there is no 
existing working traffic on any link (We = 0). We also allow the rerouted traffic to be 
split on at most two detours in the event of a failure. We show that the optimization 
problem is NP-hard and provide a 2-approximation algorithm. We also give a lower 
bound for the problem when there is no restriction on splitting of rerouted traffic. 

All the schemes mentioned earlier assume that protection is provided for a single 
link failure. Choi et al. [6] present a heuristic for protecting against two link failures, 
based on link restoration. The problem of survivable network design has also been ex- 
tensively studied [4,3]. Most of the work here has focused on obtaining strong relax- 
ations to be used in cutting plane methods. 



3 Algorithm 

Recall that Ue is the total capacity of link e, and We is the amount of existing traffic 
in link e. We define Ue — We to be the maximum protection capacity available on the 
link e. 

Our algorithm (Algorithm A) creates a solution by first computing a maximum 
spanning tree Ta based on the maximum protection capacity available on the links. 
It sets the protection capacity of any link e on the maximum spanning tree Ta to the 
maximum protection capacity available on link e. For any non-tree (Ta) link e (called 
cross link), it initially sets its working capacity equal to the minimum protection capac- 
ity assigned to the links on the unique path in Ta between the endpoints of link e. It 
then selects a few of the cross links and boosts up their protection capacities. The main 
idea is to protect the cross links by using only links in the tree Ta- Also for each tree 
link e its detour consists of all but one tree link, and the boosting up of some cross links 
is used to ensure that the cross link on this detour has enough protection to protect e. 
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The detailed algorithm is given in Algorithm A. We denote the working and protec- 
tion capacity assigned by Algorithm A on link e by wf and respectively +pf = 
Me)- Conceptually the algorithm (as well as the analysis) consists of two phases - find- 
ing basic protection capacity in all links, and then boosting the protection capacity for 
some cross links. These two steps are combined into one in the algorithm, by setting 
of cross link Ci to max(min(p, Ue^ — w), We^ ) in line 18 . p is the minimum pro- 
tection capacity of the unique path in the tree, which can protect this cross link, w is 
the maximum working capacity of a link in the unique path, which is still unprotected. 
So we need to assign at least w protection, but cannot have a working capacity of more 
than p in this cross link. Also, since we have to support all the existing traffic, w^. has 
to be at least . 

Algorithm 1 Algorithm A 

Let {ei, 62, . . . , Bm} be the links sorted in decreasing order of maximum available protection 

capacities (Ue — Ws)- 

Ta = <I> 

for i = 1, . . . , m { 

if (Ta U {ci} does not form a cycle) { 

Ta =TaU {ci} 

if {Wei > "“ei /2) 

No solution exists. 

Wi = We, 

Pt = 

Mark a as unprotected 

} else { 

Let P = {cji , 6j 2 , . . . , ejj, } be the unique path in Ta connecting the endpoints of 

6i. 

Let M be the links in P which are marked as unprotected. 

w = maxegM (ui = 0 if M = 0) 

p — mhie^PPe 

if (p < We,) 

No solution exists. 

Wei ~ max(min(p, Uei — w), Wei) 

Pt = 

if {w > pf.) 

No solution exists. 

Unmark edges in M. 

} 

} 

if any link a is marked 

No solution exists. 



3.1 Correctness 

Lemma 1. The solution (if any) returned by Algorithm A is a feasible solution. 

Proof. It is easy to see that the amount of working traffic of every link (w^) in any 
solution returned by Algorithm A, is always at least the amount of existing traffic (Wg). 
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Now we show that for every link e = {u, v), there is a backup path PeinG — e between 
nodes u and v, such that every link e' on Pe satisfies p^, > w^. 

Case 1, for all e € Ta'- Note that G—e is connected, because otherwise link e would stay 
marked at the end of the algorithm in which case no solution is returned by Algorithm 
A. Thus, there is at least one cross link Cc = (uc, Vc), which, together with the unique 
path in Ta between nodes Uc and Vc (excluding link e), forms a backup path for e. 
Without loss of generality, let Cc be the first cross link considered by the algorithm, 
such that adding it to Ta results in a cycle G containing link e. Consider the path 
Pe = C \ {e}. The link Cc has been assigned the least protection by Algorithm A 
among the links on Pe- Note that link e is marked (hence in M) at the time when Cc is 
considered by the algorithm. Thus when link Cc is considered by the algorithm we must 
have pf^ > w > We = wf implying that Pe is a valid backup (detour) for link e. 

Case 2, for all e ^ Ta' The backup path Pe for link e is the unique path in Ta connect- 
ing e, which always exists. The links on this path have enough protection because the 
algorithm sets the working traffic of link e to at most mine'g p^, . 

Lemma 2. If a feasible solution exists, the algorithm will return a solution. 

Proof The algorithm will not return a solution in 4 cases. 

Case 1, there exists a link e G Ta, Wg > Uej2: If a tree link e has Wg > Ue/2, then 
no solution exists. This is because, the maximum protection capacity available on e 
(Ug — We) is strictly less than Ug/2. If there was a solution, then there must exist a path 
Pe in G — e between the end points of e all whose links e' have maximum protection 
capacity available Ue> — Wgi > Uej2. But then Ta is not a maximum spanning tree 
based on the maximum protection capacity available on the links, a contradiction. 

Case 2, there exists a link Cc ^ Ta, p < We'. The proof for this case uses arguments 
similar to case 1 . 

Case 3, there exists a link Cc ^ Ta, w > Pg^'. Note that in this case M ^ 0. Let 
6c = (mc) t>c) and let Pg^ be the unique path in Ta connecting nodes Ug and Vg. Let 
e G M be a marked link on the path Pg^ with w = Wg = Wg, at the time when Cg is 
considered by the algorithm. Let Pg be a feasible detour for link e. Thus the maximum 
protection capacity available on all links e' on Pg is Ug' — Wg' > w. Also Pg U {e} 
forms a cycle in G, thus at least one link on Pg is a cross link for Ta, that forms a cycle 
containing link e when added to Ta. This link must have been considered before link 
Cg since it has strictly more maximum protection capacity available on it. Hence link e 
must already be marked before link Cg is considered, a contradiction. 

Case 4, there is an unmarked link e = {u, v) at the end: In this case in G — e nodes u 
and V are not connected. Hence, no detour is possible for link e. 

From the above two lemmas, we have the following theorem. 

Theorem 1. Algorithm A is correct. 

Remark: For each link e in T^i, we can lower its protection capacity until lowering it 
further would require decreasing the working capacity of some other links, or would 
make the working capacity of link e so large that there is no feasible detour for it. This 
trimming may reduce the total protection capacity however, it has no implication on our 
analysis of the worst-case approximation ratio for the algorithm. 
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4 Analysis 



In this section we show that A is a 2-approximation algorithm. First we define some 
notations. Let be the protection capacity of link e in an optimal solution OPT, 

and be the working capacity of link e in OPT, thus = Ue — To 

compare the solution of Algorithm A with OPT, we construct a maximum spanning 
tree Tqpt based on the protection capacities p^^^. 

We partition the links in the network into 4 sets: Eao = {e|e G Ta, e G Tqpt}, 
Eao = { s |6 ^ Ta,g ^ Tqpt}, Eaq = { c|c G Ta,g ^ Tqpt}, snd Eaq = {e|e ^ 
TA,e G Tqpt}- We define to be (sum of protection capacities 

in OPT over all links in E), p^ to be (sum of protection capacities in our 

solution over all links in E), and up to be Ug. For a link e ^ Ta, we use to 

represent the unique path in Ta connecting the endpoints of e. Similarly for e ^ Tqpt, 
we use Bg^"’" to represent the unique path in Tqpt connecting the endpoints of e. 
Since both Ta and Tqpt are spanning trees, both trees have the same number of edges. 
It is easy to see that = \Eao\- 

For a cross link e (not in Ta), we assume that the Algorithm A assigns a basic 
“Level-1” protection capacity of Pg^ = max(0,Ue — miUg/ggAp^) and a working 
capacity of Wg^ = min(ue, ming/g^A p^). The cross links e with Pg ^ = p^ are non- 
boosted cross links. Note that the protection capacity of the remaining cross links e 
are boosted from Pg ^ to Wgi by Algorithm A, where e' is the tree link with the largest 
existing traffic {Wg/), among all tree links protected by e (which are in M when link e is 
considered by the algorithm). Also note that there are at most n — 2 such boosted cross 
links, each protecting a different link in Ta- It is easy to see that p^ < YheSiTA 
Pe^ ■*” SeGTA 

The total protection capacity used by Algorithm A is thus at most p^^^ + Pp _ + 
Pe\o Pe\o ^Eao + ^Eao ■ 

Lemma 3. For all links Cao € Eao< Wg^^ < pS^"’" , for any e' G B^opt- 

®AO 

Proof. For all links Cao € (note that Cao is a cross link for Tqpt) 



..OPT 



< • \ 

min n 

a AO — 



OPT 



ees ' 



< mm V 
e^popp 
®AO 

^OPT 



where S is the backup (detour) path for ^Ao OPT 
OPT 



^ JJPT g o' c R 
S Pg' , for any 6 G 



OPT 



( 1 ) 

( 2 ) 

( 3 ) 



The first inequality follows from the definition of S. The second inequality holds be- 
cause To PT is a maximum spanning tree of OPT based on the protection capacities 
P?^^- 

Lemma 4. For all links bao ^ P^AO’ pfP ^ ming^pA pf. 

AO „AO 

Proof Note that cao is in the OPT tree Tqpt, but not included in Ta- Therefore the 
maximum protection possible on cao in any solution is at most the minimum protection 
assigned by Algorithm A to the links on the unique path in Ta connecting the endpoints 
of link cao- 
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Lemma 5. For any links G and any links e^Q € Eao where e^o 

on the unique path in Tq pt connecting the endpoints of link Cj^q, we have < 
OPT 

Ue — wf . . 

^Ao 

Proof. 



Pelo = ™ax(0, - mirieeBf. pf) 



< u 

< u 



^Ao 



^AO 



-P 



OPT 

^Ao 



— W 



OPT 

^AO 



(4) 

(5) 

(6) 



The first equality follows from the definition of Pe^^ ■ The second inequality holds by 
Lemma 4 and the fact that > pfP- The third inequality holds by Lemma 3, with 
e' = e^o- 



Lemma 6. For any links Cj^q G E^q, and any links e^o ^ E^q where e^o on 

the unique path in Tq pt connecting the endpoints of link Cj^q, we have p^ _ + + 

Pe- < P?^^ + 

Proof 



Pto 



VLe 



' Pe- ^ 



Ue 



AO 
pPT 
AO 

< P?^^ 

— ^^AO 



= Pe 



+ K.o-<7) 


(7) 


' + '^eAo 


(8) 


' + ^Pelo 


(9) 



The first inequality follows from Lemma 5. The second equality follows from the defi- 
nitions of protection and working traffic. The third inequality follows from the fact that 
must be at least Ue/2 for any link e € Tqpt- The proof is along the same lines 
as the proof of Case 1 in Lemma 2. 

Lemma 7. p^^^ + +Pe\o - Peao + ‘^Peao' 

Proof For each link G ^ao> show how to pair it up with a unique link 

e^O S E^q, where e^o li®s on the unique path in Tqpt connecting the endpoints of 
link 6j^q. Summing Lemma 6 over all such pairs, the lemma follows. 

Consider a bipartite graph {Ej^qUE^q, Em), where Em contains edge (e^g, cao) 
if Cao 1i 6S on the unique path in Tqpt connecting the endpoints of link A one 
to one pairing of edges in E^^q with edges in E^q can be done by finding a perfecf 
matching on the bipartite graph. We now show a perfect matching must exists, using 
Hall’s Theorem [21]. Given any subset S of Ej^q, find a forest E which is the union 
of all the unique paths in Tqpt connecting the endpoints of links in Ej^q. Note that 
all links in F are in Tqpt- It is easy to see that at least 151 links in F are not in Ta- 
Otherwise we can create a cycle in Ta involving a link in S and links of F which are 
in Ta- Therefore links in S may be paired up with at least |5| links in Eaq, and the 
condition in Hall’s Theorem is satisfied. 



Theorem 2. A is a polynomial time 2-approximation algorithm, i.e., Pp^o Pe - E 
pj^\^ + p%\^ + < 2pOP^. 
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Proof. From Lemma 7 we have + Pe\o - + ‘^Peao' 

Peao ~ "^Eao — ‘^Peao OPT has to reserve at least i the capacity of 

each link in Eao for protection (see Lemma 6). Also < Peao f*®oause there is 

no solution in which the working capacity of a cross link cao can exceed ~PeAo ' 
The proof follows. 

4.1 Worst Case Example 

We give a worse case example to show that the analysis is almost tight. Consider a ring 
of size n with unit capacity, the optimal solution is to put 0.5 units of protection on each 
link. This gives a total protection of ^ units. Our algorithm would return a tree of size 
n — 1 and put full protection to each link, which gives a total protection of n — 1 units. 
Therefore, the approximation factor is close to 2. 



5 LP-Based Approach 

In this section we study a natural relaxation to our problem which can be modeled as a 
linear program. In this relaxation on the failure of a link its traffic can be arbitrarily split 
and rerouted over multiple detours. That is, in this relaxation we require that the graph 
G — e must support at least We flow between nodes u and v, for every link e = (u, v), 
using only the protection capacities on the remaining links. The LP is shown below. 

For the LP, we transform the undirected graph into a directed graph by replacing 
each link into two directed links. We use the notation pair{e) to denote the link in the 
opposite direction of link e. We assume that both the directed links fail together. Note 
that in this relaxation on the failure of link e, at most We traffic on it is rerouted over 
multiple detour paths using only the protection capacities on the links on the paths 
Pe- We use flow{e, i) to denote the amount of flow rerouted on link i on the failure of 
link e. The relaxed LP is 



Minimize Y.eeE(Pe/‘^) 

Subject to the following constraints 



flow{e,i)= 


flow{e,i) \/e£E,vGV 


(10) 


flow{e,pair{e)) = We 


ye eE 


(11) 


flow{e, e) = 0 


ye£E 


(12) 


flow{e,i) < Pj 


Ve, i G E,e pair{i) 


(13) 


We > We 


WeGE 


(14) 


Pe + We= Ue 


WeGE 


(15) 


Wq — '^pair{e) ^ ^ 


(16) 


Note that since the protection capacity of each (undirected) edge is counted twice, once 
for each direction, we divide the total protection capacity by 2 in the objective function. 
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Constraint 10 takes care of the flow conservation at each node (including the end nodes 
of e). For link e = (u, v), a flow of We from node u to node v is guaranteed by sending 
a flow of We from t; to it (constraint 11) and using the flow conservation equations at 
nodes u and v. Constraint 12 guarantees that this flow conservation is not achieved using 
the link (it, v). Constraint 13 guarantees that the flow does not exceed the protection 
capacity of any link. Constraint 14 guarantees that the working capacity of a link is at 
least the existing traffic. Constraint 15 guarantees that sum of protection and working 
capacity does not exceed the total capacity. Constraint 16 is for guaranteeing a solution 
for the undirected graph. 

The results of [ 1 ] imply that this LP has an integrality gap of 2 when the existing 
working traffic We on all links is zero. We show however that when We is not restricted 
then this gap is Q{n). The proof of the following lemma appears in the full paper [2]. 

Lemma 8. The integrality gap of the LP or the ratio of the optimal integral solution to 
the optimal relaxed solution is fl(n). 



6 NP-Hardness 

In this section we show using a reduction from 3-SAT that the decision version of the 
problem to pre-provision existing networks to support fast restoration with minimum 
over-build (PREPROVISION) is NP-hard. 

Let {Xi,X 2 , . . . , Xn} be the set of variables and {Ci, (72, . . . , Cm} be the set of 
clauses in a given instance C of 3-SAT. We construct an instance of PREPROVISION 
such that it has a feasible solution of total protection capacity at most 4.5n + 24m if 
and only if C is satisfiable. As shown in Eigure 1, for each variable Xi we create a 
ring consisting of 6 unit-capacity links (arranged like a hexagon) without any existing 
traffic. We have a unit-capacity link splitting the ring into two equal halves. This link, 
with existing traffic of 1 unit, is represented by a thick segment in Eigure 1 . Three of 
the links, on the left part of the ring, correspond to literal Xi, and the other three links, 
on the right part of the ring, correspond to literal Xi. For each clause Cj we create 
an octopus-like structure. In the center we have a unit-capacity link Ic with existing 
traffic of 1 unit (represented by a thick segment in Figure 1). Three pairs of feet, where 
each pair corresponds to a literal in the clause, are attached to the two endpoints of link 
ICj ■ Each foot consists of 6 serially-connected unit-capacity links without any existing 
traffic (represented by a dash segment). If literal Xi appears in clause Cj, one foot 
connects the upper endpoint of link Ic^ to the node between the first and second link 
of the half-ring corresponding to Xi. The other foot of this pair of feet connects the 
lower endpoint of link Icj to the node between the second and third link of the half-ring 
corresponding to Xi. Therefore, we have 6n + 32m nodes and 7n -f 37m links in the 
network. 

Note that if each foot in the octopus consists of only two links (rather than 6 links), 
we can still prove that the 3-SAT instance is satisfiable if and only if there exists a feasi- 
ble solution to the PREPROVISION instance with total protection capacity of 4.5n-|-8m 
(with 6n + 8m nodes and 7n + 13m links). However the analysis is much more com- 
plicated and thus we present the proof with 6 links. 
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Cl = Xi V X2 V c^ = Xi V X2 V 

Fig. 1. Gadgets 



Lemma 9. If the 3 -SAT instance is satisfiable, then there exists a feasible solution to 
the PREPROVISION instance with total protection capacity ofA.bn + 24m. 

Proof If a literal Xi{Xi) is true in the satisfying assignment, we put protection of 1 
unit on each of the three links in the half-ring corresponding to Xi{Xi). Otherwise we 
put protection of half units on each of these three links. Therefore a total of 4.5 units of 
protection is reserved for each variable. In each clause, one of its literals must be true. 
We put protection of 1 unit on each of the 12 links between the clause and one of the 
true literals. We put protection of half units on each link in the remaining two pairs of 
feet (each pair of feet consists of 12 links). Therefore a total of 24 units of protection is 
reserved for each clause. For links with an existing traffic of 1 unit (the thick segments), 
we have no choice but to put 0 units of protection. The total protection capacity reserved 
is therefore 4.5n + 24m. 

It is easy to see that with this reservation all links are protected by a detour path. The 
link which splits a ring in a variable is protected by the 3-link half-ring corresponding 
to the true literal for the variable. Since there is at least one true literal in each clause, 
the link Ic^ in the middle of the gadget of a clause is protected too. The working traffic 
in the remaining links is either zero or half, and therefore they can be protected because 
all the remaining links have protection capacity of half units. 

Due to lack of space, we give the proof of the other direction in the full paper [2]. 

Theorem 3. The problem of pre-provisioning existing networks with minimum over- 
build is NP-hard. 
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Abstract. We consider the problem of Simultaneous Source Location - 
selecting locations for sources in a capacitated graph such that a given 
set of demands can be satisfied. We give an exact algorithm for trees and 
show how this can be combined with a result of Racke to give a solution 
that exceeds edge capacities by at most 0(log^ nloglogn), where n is 
the number of nodes. On graphs of bounded treewidth, we show the 
problem is still NP-Hard, but we are able to give a PTAS with at most 
O(H-e) violation of the capacities, or a (fc + l)-approximation with exact 
capacities, where k is the treewidth and e can be made arbitrarily small. 



1 Introduction 

Suppose we are given a capacitated network and we have various demands for 
service within this network. We would like to select locations for servers in order 
to satisfy this demand. Such problems arise naturally in a variety of scenarios. 
One example of this would be the placement of web caches in the Internet, or 
file servers in an intranet. Another example would be choosing the locations of 
warehouses in a distribution network. 

What does it mean for a server to “serve” a demand? Most previous work 
has assumed that each server can service each demand for some cost (typically 
this cost is linear in some underlying distance between server and demand) [20, 
16,3]. In some cases the servers have been considered to be capacitated (each 
one can provide for only some number of demands) [17]. Still, the primary goal 
can be considered as minimizing the aggregate distance of demands to servers. 

In many natural applications, there is no meaningful notion of distance. Con- 
sider serving a web page across the internet. The latency (travel time of a single 
small packet) under low-congestion conditions tends not to be noticeable to the 
end-users. The real difficulty here is the underlying capacity of the network. If 
links become congested, then latency will increase and throughput will suffer. In 
the case of a distribution network, there may be some relation (typically non- 
linear) of costs to distance traveled. But we will definitely have to consider the 

* The authors want to thank the Aladdin Center. This work was in part supported by 
the NSF under grants CCR-0085982 and CCR-0122581. 
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available throughput! The transportation network has capacities as well as costs, 
and previous work (assuming costs only) tended to ignore this constraint except 
at the warehouses themselves. 

We consider the problem of Simultaneous Source Location (SSL) - selecting 
locations for sources in a capacitated network in order to satisfy given demands. 
Our goal is to minimize the number of sources used. Arata et al. previously gave 
an exact algorithm for the Source Location problem, in the scenario where the 
sources must be able to satisfy any single demand [2]. They also show that the 
Source Location problem is NP-hard with arbitrary vertex costs. In our problem 
we must satisfy all demands simultaneously (thus the name Simultaneous Source 
Location). This is a better model of the natural problems described above. Si- 
multaneous Source Location is easier than the Source Location problem with 
arbitrary vertex costs explored by Arata et al. - our version of the problem can 
be reduced to theirs using a high-cost super-sink that is satisfied if and only if 
all other demands are met. However, we show that our version of the problem is 
NP-Hard as well and describe various approximations for it. 

Our results take several forms. We describe techniques for solving Simultane- 
ous Source Location on a variety of simple graphs. We give an exact algorithm 
on trees and an approximation for graphs of bounded treewidth. We observe 
that, in contrast to many other NP-Hard problems (for example vertex cover). 
Simultaneous Source Location is still NP-Hard even on graphs of treewidth two. 
Using our algorithm for trees combined with a result of Racke et al. [18, 6] and 
later Harrelson et al. [11], we show how to solve source location on a general 
undirected graph while overflowing the capacity by an 0(log^ n log log n) factor. 
Combining this with a hardness result for directed graphs allows us to show 
that no tree decomposition similar to Racke’s decomposition can be found in 
the directed case. We show Simultaneous Source Location is at least hard on 
general undirected graphs, but there remains a significant gap between the best 
approximation algorithm and the known lower bound. 

2 Problem Statement 

An instance of Simultaneous Source Location (SSL) consists of a graph G = 
(V,E) along with a capacity function u : E ^ on the edges and a de- 
mand function d : V ^ on the vertices. We must select some subset of the 
vertices S' C U to act as sources. A source set is considered to be feasible if 
there exists a flow originating from the nodes S that simultaneously supplies 
demand dy to each node v G V without violating the capacity constraints. This 
is single-commodity flow - we can imagine adding a single “super-source” that is 
connected to each node of S with an infinite capacity edge, and a single “super- 
sink” that is connected to each node of V with an edge of capacity , and asking 
whether the maximum flow equals the sum of the demands. Our goal is to select 
such a source set S of the smallest size. 

From the above description, it is clear that the problem is in NP. A source set 
S can be checked for feasibility by simply solving a flow problem. Unfortunately, 
finding the set S of minimum size is NP-Hard. 
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At times we consider various generalizations of this problem. For example, 
we might have certain nodes in V that are not permitted to act as sources, or 
have costs associated with making various nodes sources and seek to find a set S 
of minimum cost. We will also consider simplifications of the problem that place 
restrictions on the graph G (for example by bounding the treewidth). 



3 Solving Simultaneous Source Location on Trees 

Suppose our graph G = (V, E) is a tree. This special case allows us to solve SSL 
exactly by dynamic programming. For each vertex v and number of sources i, 
we define f{v, i) to be the amount of flow that must be sent to v by its parent in 
order to satisfy all demands in the subtree of v, assuming this subtree contains i 
sources. Our algorithm assumes that the tree is rooted and binary; in general we 
can create an equivalent binary tree by creating virtual nodes and connecting 
them by edges of infinite capacity, as shown in Figure 2. For convenience, for 
each node v we define u{v) to be the capacity of the edge from v to its parent. 

Our algorithm is described in Figure 1. 



Algorithm Binary Tree(G,d) 

1. Initialize f{v,i) = oo for all w € F and 0 < i < |F| 

2. For each leaf vertex v G V: 

(a) Set f{v, 0) = 

(b) Set f {v,i) = —u{v) for all i > 1. 

3. Consider any vertex v with children vi , V 2 for whom / has been computed: 

(a) Loop over all values of ii and i 2 with 0 < *i ,*2 < \V\. 

(b) If f{vi,ii) < u(vi) and /(i> 2 ,* 2 ) < u(v 2 ) then: 

i. Set f(v,ii+i 2 ) = min(f(v,ii+i 2 ),max(f(vi,ii) + f(v 2 ,i 2 )+dv,—u(v))). 

ii. Also set f(v, ii + h + 1) = —u(v). 

4. Continue until / is defined at all vertices. 

5. Return the minimum k such that /(r, fc) < 0 where r is the root. 

Fig. 1. Algorithm for SSL on a Binary Tree 



Assuming that the above algorithm computes f{v, i) correctly for each vertex, 
the correctness of the result is immediate. We need to show that /(n, i) correctly 
represents the minimum amount of flow that must be sent from the parent of v 
provided the number of sources in the subtree is i. 

Theorem 1. The algorithm described produces an exact solution to the source 
location problem on a binary tree. 

Proof. The proof will be by induction. Our base case is at the leaves. Either a 
leaf is a source or it is not. If the leaf u is a source, then it requires no flow from 
its parent, and can send at most u{v) flow upwards along its edge. This yields 
f{v, 1) = —u{v). On the other hand, if the leaf is not a source it requires flow 
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(its demand) from the parent, so f{v, 0) = dv Of course, it might not be feasible 
to send this amount of demand if > u{v). 

We now consider any node v. Suppose we have correctly computed /(ui,ii) 
and /(u 2 )* 2 ) for all values ti,Z 2 for the children of node v. Suppose we would 
like to compute f{v,i). There are i sources to be placed in this subtree. If v is 
not a source itself, then all the sources are in the child trees. The total demand 
sent into v will have to be enough to satisfy the demand and additionally to 
satisfy any residual demand on the children. This means f{v, i) = min(max((i„ + 
/(ui, ii) + f{v 2 ,i — ii), —u{v))) where the minimum is over choices of ii and the 
“max” term ensures that a child node cannot provide more flow to its parent 
than the capacity of their connecting edge. This is exactly what will be computed 
in the algorithm, in step 3b(z). Notice that if satisfying either child tree in this 
way would require overflowing a capacity u{vi) or u(v 2 ) then this allocation 
of sources to subtrees is not feasible and so should not be considered; this is 
resolved in step 3b. However, it is also possible that v is itself a source. In this 
case, provided there is some choice of ii < z — 1 such that f{vi,ii) < u{vi) and 
f{v 2 , i — ii — l) < u{v 2 ), we will be able to produce a solution. This solution can 
send u{v) upwards since v itself is a source. This is dealt with in step 3b(ii). It 
follows that the algorithm computes exactly the correct values f{v,i) and the 
algorithm solves SSL. 

Theorem 2. We can solve SSL in time O(n^) on a tree, even if some nodes 
are disallowed as sources. 

If our tree is non-binary we can replace any node with more than two children 
by multiple nodes as shown in Figure 2. This increases the number of nodes by 
at most a constant factor. 




Fig. 2. By adding additional nodes linked by infinite capacity, we can convert any 
non-binary tree into an equivalent binary tree by only doubling the number of uodes 
in the tree 



We can also modify our algorithm to disallow certain nodes as sources. If a 
leaf V cannot be a source, then we ignore step 2b and instead set f{v, i) = for 
all i. If some higher node cannot be a source, then we remove step 3b(ii) for that 
node (this step considers the case where u is a source). The correctness proof is 
the same as before. 




Simultaneous Source Location 



17 



4 Using Racke’s Result 

In recent work, Harold Racke showed that for any undirected graph, it is pos- 
sible to construct a tree which approximately captures the flow properties of 
the original graph[18]. Later work [6,11] improved the result and described a 
polynomial-time algorithm to construct such a tree. The most recent result is 
stated more precisely in the following theorem: 

Theorem 3. Given any capacitated, undirected graph G = (V, E,u : E ^ R'^), 
there exists a capacitated tree T = {Vt,Et,u : Et R~^) with the following 
properties: 

1. The vertices of V are the leaves ofT. 

2. For any multicommodity flow E which is feasible on G, there exists a flow 

of equal value between the corresponding leaves on T. 

3. For any flow Ft feasible between the leaves ofT, there is a feasible jFt flow 

on G for some p = 0(log^ n log log n). 

This gives us an approximation for the source location problem. We first 
construct a tree T as described above. We then solve SSL on the tree, permitting 
only the leaves to act as sources, using the algorithm of Section 3. We consider 
using the sources we have obtained on the original graph. We know there exists 
a flow Ft on the tree from our selected source nodes which satisfies all the 
demands. It follows that ^Ft is feasible on the graph. We conclude that if we 
violate the capacities by a factor of p = 0(log^ nloglogn), then we have a 
feasible solution to source location. On the other hand, any feasible solution on 
the graph must also be feasible on the tree. This allows us to produce an exact 
solution (in terms of the number of sources) while increasing the capacities by 
0(log^ nlog log n). 

Theorem 4. We can produce an optimum (in terms of number of sources) so- 
lution to SSL in polynomial time, if we permit 0(log^ nloglogn) stretch on the 
capacities. 

Our results also have interesting implications for directed graphs. Consider 
the possibility of a Racke-like representation of a directed graph by a tree, where 
the capacity on an edge when routing “upwards” might differ from the “down- 
wards” capacity. Suppose such a thing existed, and could be computed in poly- 
nomial time, for some factor p. This would enable us to solve SSL exactly on 
directed graphs, while exceeding capacities by a factor of p. But this is NP-Hard, 
as will be shown in Section 6.2. Thus we have the following: 

Theorem 5. No Racke-like decomposition of a directed graph into an p-appro- 
ximately flow-conserving tree can be computed in polynomial time, for any value 
of p polynomial in n. 

Note that our hardness result is a computational hardness result and assumes 
a tree-like structure decomposition. Azar et al. [5] have found an existential 
hardness result which is true for any decomposition, but their bound is weaker: 
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5 Simultaneous Source Location with Bounded Treewidth 

5.1 Defining Treewidth 

The notion of treewidth was introduced by Robertson and Seymour [19]. Many 
problems that are in general intractable become polynomial-time solvable when 
restricted to graphs of bounded treewidth. Furthermore, many graphs arising 
from natural applications have bounded treewidth. A good survey on the topic 
is given by Bodlaender [7]. Here is one of the many equivalent definitions of 
treewidth: 

Definition 1. A graph G = (V, E) has treewidth k if there exists a tree t = 
(V-rjEr) along with a mapping f : Vr ^ 2^ with the following properties: 

1. For all a G Vr, |/(o:)| < fc -I- 1. 

2. For any (u,v) G E there exists some a G Fr such that u,v € f{a). 

3. For any a, (3,^ G Vr where j3 lies along the path from a to 7, if for some 
X GV we have x G f{a) and x G /(y), then we must also have x G /(/3). 

The above conditions essentially state that each tree vertex represents some 
subset of at most k graph vertices, each edge in the graph has its endpoints 
represented together in at least one of the tree vertices, and the set of tree 
vertices which represent a single graph vertex must form a contiguous subtree. 

We observe that it is possible to produce such a tree decomposition for a 
graph of treewidth k in time linear in the number of nodes and vertices (but 
exponential in k). Assuming k is constant, we are able to produce a tree de- 
composition - and thereby implicitly detect graphs of constant treewidth - in 
polynomial time. In general, however, computing the treewidth of an arbitrary 
graph is NP-Hard. 



5.2 Nice Decompositions 

Bodlaender [ 8 ] also introduced the notion of a nice decomposition and proved 
that any tree decomposition of a graph can be transformed into a nice decom- 
position still of polynomial size. In a nice decomposition, each node a G Vr has 
one of the following types: 

— A leaf node a has no children, and |/(q;)| = 1 

— An add node a has one child fd with f{a) = f{f3) lj{^} for some node v GV 

— A subtract node a has one child f3 with /(a) = /(/?) — {u} for some node 

V G /(/3) 

— A merge node a has two children /3 , 7 with /(a) = /(/?) = /(y) 



In addition, the nice decomposition has a root node p (which is a subtract node) 
with /(p) empty. 
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5.3 Approximation for Graphs of Bounded Treewidth 

Suppose we are given a graph G = (V, E) with treewidth k, for which we would 
like to approximate the SSL problem. Our algorithm takes in some set of current 
sources S along with a graph {V, E) and returns a set S' of sources which are 
feasible for the given graph. Our algorithm for this problem appears in Figure 3. 



Algorithm SL(S,V,E) 

1. Check whether the source set S is feasible for {y,E)\ if so return S 

2. If not, find sets X and Bx that have the following properties; 

(a) \Bx \ < fc + 1 

(b) For all (x, y) € E with x € X and y € V — X, we have x € Bx 

(c) S'U(k^ “ Is not a feasible source set 

(d) S'U(0 — U Bx is a feasible source set 

3. Recursively solve SL{S\J Bx,{V ~ X)\J Bx,E) 

Fig. 3. Algorithm for SSL with treewidth k 



We claim that the set of returned sources has |S''^| < {k + 1)|<S'*| where 
I S'* I is the smallest feasible set of sources for (V,E) which includes the given set 
S as a subset. 

Lemma 1. Assuming we are always able to find sets X and Bx with the prop- 
erties described, algorithm SL is a {k-\- 1)- approximation for the source location 
problem. 

The prove is done by induction on the number of sources required by the opti- 
mum solution. 

Of course, we still need to prove that we can always find the sets X, Bx with 
the required properties in polynomial time. In a general graph, such a pair of 
sets might not even exist, but we will use the assumption of treewidth k to prove 
existence and the ability to find the sets in polynomial time. 

Lemma 2. If the current set of sources S is not feasible for the graph G = 
(y, E) with treewidth k, then there exists a pair of sets X, Bx with the required 
properties; furthermore, such a pair of sets can be found in polynomial time. 

Proof. We produce a tree decomposition (r, /) of G. For each tree node a, we 
define to be the subtree of r rooted at a. We define /(tq) = U/ 3 gtc, fW- For 
each node a we will test whether 5 lj(y — /(tq,)) is a feasible set of sources. We 
find node a such that S[j{V — f{Ta)) is infeasible, but such that S[j{V — f{Tjs)) 
is feasible for each child (3 of a. Note that such an a must exist; we simply 
travel upwards from the each leaf of the tree until we find one. We now consider 
returning X = /(tq.) and Bx = f{ct). We will show that these sets satisfy the 
required properties. 
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Since the graph has treewidth k we know \Bx\ = |/(ct)| < k+1. Consider any 
(x,y) G E with x € X and y € V — X. From the definition of treewidth, there 
must exist some node (3 with x,y € /(/3). Since y is not in /(tq) we conclude 
that j3 is not in Tq,. On the other hand, there is some node ^ & Ta such that 
X £ f{'j) (this follows from x £ X). The path from 7 to /? must pass through a, 
so the treewidth definition implies x £ f{a) and therefore x £ Bx as desired. 
The selection of a guarantees that S'U(^ ~ is ^ feasible source set. This 
leaves only the fourth condition for us to prove. 

We consider the children of a. A pair of them 71, 72 must have /(71) H /(72) 
G /(a) because of the contiguity property of tree decomposition. Thus the sets of 
nodes represented by the subtrees of the children can intersect only in the nodes 
of Bx = f{a)- We know that for each child 7, the set of nodes S'U(^ ~ /('^i)) 
would be feasible. It follows that we can cover all the demand of /{tj) using 
nodes of S and nodes external to the set. Since the children sets are disjoint 
except for nodes which we have declared to be sources, the flows to satisfy each 
child subtree are edge-disjoint; any flow from external nodes must also pass 
through Bx, and we conclude that we can cover /(r^) using S[jBx- It follows 
that we can cover all of X using the sources S IJ Bx, making S IJ Bx U(^ ~ 
a feasible source set for the entire graph. 

Theorem 6. Algorithm SL produces a k + 1- approximation to the SSL problem 
on a graph of treewidth k . 



5.4 Bounded Treewidth with Capacity Stretch 

We will describe an exact algorithm for the SSL problem on graphs of bounded 
treewidth. The running time of this algorithm will be exponential in the tree- 
width, and also depends polynomially on the maximum edge capacity. In the 
general case where the capacities may be large, we will use the technique of 
Appendix A to obtain a solution with 1 -|- e stretch on the edge capacities. 

Suppose we have found a nice tree decomposition (r, /). We will construct a 
set of vectors of dimension fc -I- 3. Each vector has the following form: 



{oi,i,h,f2,---,fk+i) 



Here a £ Vt, < i < \V\, and the fi are feasible fiow values (we assume 
these are from a polynomially-bounded range of integers). Let Sa = f{xa) — f{a) 
represent the nodes represented by the subtree rooted at a minus the nodes of 
its boundary (the nodes of /(a) itself). A feasible vector represents the excess 
fiow needed to be sent directly from the nodes of /(a) to Sa to satisfy all the 
demand in Sa if Sa contains i sources. 

We observe that if fc is a constant and fiow is polynomially-bounded, then 
the number of possible vectors is polynomial in size. We will determine which 
such vectors are feasible in polynomial time, then use this to solve SSL. Our 
algorithm is as follows: 
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1. Start with the empty set of feasible vectors. 

2. Consider each node a from the bottom up: 

— If a is a leaf, then add vector {a, 0, 0). 

— If a is an add node with child /?, then for each feasible vector for /?, copy 
that vector for a, placing flow 0 in the new position corresponding to 
the additional node in /(a) — 

— If a is a merge node with children (3, 7 then for each pair of feasible 
vectors xp,x-f for the children, create a vector for a: Xa = Xjs + x-y 
(adding the number of sources and the flows while changing the choice 
of node from Vr to a). 

— If a is a subtract node with child P, then consider each feasible vector 
X/s for the child. Let the subtracted node be 5 G /(/?) — f{a). This node 
requires some flow rf, which is the sum of the demand db and the flow 
value for b in x/ 3 . We consider all feasible allocations of flow F{a) to 
nodes a G f{a) such that |i^(a)| < u{b,a) and X)ae/(a) 
each such allocation we construct a vector Xa whose number of sources 
is equal to and with flow value at a equal to the flow value in Xfj plus 
F{a). This corresponds to refusing to make b a source. We now consider 
making b a source. This corresponds to creating a vector Xa with one 
more source than the vector Xjs. We set the flow value for a node a G f{a) 
to be the flow value in x/s minus u{b, a). 

3. Now consider all vectors for the root node p. These are simply pairs (p, i) 
since f{p) is empty. We return the minimum value of i such that (p,i) is in 
the feasible set. 

Theorem 7. If there are F possible flow values, the above BTW algorithm runs 
in time 0{kn^ N F'^^~^'^) where n = \ V\ and N = \Vr\ and k is the treewidth. 

This running time is polynomial assuming that F is polynomial and A: is a 
constant. If the number of possible flow values is not polynomial, we can use the 
result of Appendix A to effectively reduce the number of flow values. This will 
cause us to exceed the graph capacities by a factor of 1 + e. 

5.5 Lower Bound for Treewidth 2 Graphs 

We show that the SSL problem is NP-Hard even on graphs with treewidth two. 

Theorem 8. SSL is NP-Hard even on graphs with treewidth two. 

Proof. The proof is by reduction from subset sum. We are given a set of numbers 
{xi,X 2 , ■ . ■ , Xn}, and would like to determine whether some subset of the given 
inputs sums to A. Suppose the sum of all the numbers is S. We construct an SSL 
instance with 2n + 2 nodes. Each number will be represented by a pair of nodes 
of demand S with an edge of capacity S between them. Our example is a four 
level graph. On the first level we have a node of demand A which connects to one 
side of every pair of nodes that represent a number. The capacity on the edge 
between the A node and the number Xi node is exactly Xi. The number nodes 
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Fig. 4. Simultaneous Source Location is NP-Hard even on graphs of treewidth two. 
To satisfy each demand in this graph using only n sources we must find a partition of 
{xi,X2, • • • , Xn} whose sums are A and S — A 



from the second level are paired with the nodes from level 3. All nodes from level 
3 are connected to a single node at level 4 with an edge of capacity corresponding 
to their number. The node at level 4 has demand S — A. This graph is shown 
in Figure 4. If there exists a subset of the numbers summing to A, then we 
can place sources on the lefthand node for each of the numbers in that subset 
and the righthand node for all the other numbers; it is straightforward to see 
that this is a feasible SSL making use of n sources. On the other hand, consider 
a SSL solution. We must select one of the two nodes for each of the numbers 
(otherwise there is not enough capacity to satisfy them) . It follows that the SSL 
uses at least n sources. If exactly n sources are used, then the result corresponds 
to a subset sum solution. It follows that solving source location exactly on this 
graph will solve subset sum. The graph given has treewidth two; we can see this 
because if we remove the node of demand A, the remaining graph is a tree. We 
take the (treewidth one) tree decomposition and add the node of demand A to 
the subset f(a) for all a. This is a tree decomposition of width two. 



6 Simultaneous Source Location on Directed Graphs 

6.1 Greedy O(logn) Approximation 

We are given a directed graph G = (V, E) for which we would like to approx- 
imate the SSL problem. We propose a simple greedy algorithm. We start with 
no sources and no demand satisfied. We add the source which maximizes the in- 
crease in the total satisfied demand. We repeat this until all demand is satisfied. 

Theorem 9. The greedy algorithm gives an 0(log n) approximation on the num- 
ber of sources with no violation of the capacities. 

Proof. Suppose that the optimum solution uses t sources. At some point in time, 
suppose our current set of sources can cover demand d and the total demand 
in the graph is D. Consider the residual graph after adding the flows from our 
sources to satisfy demand d. The residual demand is D — d, and if we were to 
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add all the optimum sources we would be able to satisfy the full residual demand 
(note that this is essentially single commodity flow since sources are equivalent) . 
It follows that we can add one source to the residual graph to satisfy demand 
-2^. We apply the standard greedy analysis for problems like SETCOVER to 
show that the full demand will be covered in 0(log D) steps. Assuming that the 
maximum and minimum demand are within a polynomial factor of one another, 
we are done. Otherwise, we can apply scaling arguments of Appendix A to give 
the desired 0(log n) factor. 



6.2 Lower Bound for Directed Graphs 

We show that 0(log n) is the best approximation factor we can expect to obtain 
for the directed SSL problem in polynomial time, due to a reduction from set 
cover. 

Theorem 10. We cannot approximate directed SSL to better than O(logn) in 
the worst case, unless NP C DTIME{n'^^^°^^°^^^). 

Proof. Suppose we would like to solve a set cover instance. We construct a graph 
with one node for each set, one node for each element, and one additional node. 
The additional node is connected by a directed edge of capacity one to each of 
the sets. Each set is connected by a directed edge of capacity N, where N is 
more than the sum of the number of sets and elements, to each of its elements. 
The additional node has demand one, each set has demand one, each element 
has demand N . We solve the SSL problem to some approximation factor p on 
this graph. We first observe that no element should be selected as a source; 
otherwise we could simply select one of the sets containing that element as a 
source instead. Second, we observe that the additional node will be selected as 
a source. We have a solution consisting of pt nodes, where t is the optimum. 
Consider the set nodes that we selected as sources. Every element node must be 
connected to one of these set nodes in order for its demand to be satisfied (note 
N is greater than the number of sets). It follows that we have a set cover of 
size pt—1. Similarly, observe that any set cover, plus the additional node, forms 
a feasible SSL solution. So we have obtained a ^ P approximation to set 
cover. This is unlikely for p smaller than logn due to the results of Feige [10]. 

We observe that even if we are allowed to violate edge capacities by some 
large factor (say less than N), the reduction from set cover still holds. 



7 Lower Bound for Undirected Graphs 

We show that SSL does not have a polynomial-time approximation scheme via 
an approximation-preserving reduction from vertex cover. 

Assume we can approximate SSL on any graph to a constant a. Now for an 
instance of Vertex Cover on a graph G=(V,E), we will setup an instance of SSL. 
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For every vertex the demand is equal to its degree and all edge capacities are 
unit. 

The main observation is that a vertex cover set and a source location set 
on this graph are equivalent. It is easy to see that a feasible vertex cover is a 
feasible source location: for every edge at least one end is in the vertex cover 
and therefore a source. Thus every vertex will have its demand satisfied. On the 
other hand a source location set can’t have a two hop path over a non source 
vertex, because this vertex will be starved. Hence every edge has at least one end 
in the source location set, i.e. the source location set is a feasible vertex cover. 

Using the assumption we can approximate the SSL problem on G. Therefore 
we find a set ScV which covers all edges in E and is within a of the optimal 
Vertex Cover. However as Hastad [12] showed Vertex Cover is inapproximate if 
a <1 jQ (although an approximation factor better than 2 would be a surprising 
improvement on existing vertex cover results) . 

Theorem 11 . Simultaneous Source Location is 1.36067— e hard on general undi- 
rected graphs if edge capacities are not violated. 

The proof follows from the reduction above and the recent hardness results 
by Dinur and Safra [9] . 

8 Conclusions 

We define the Simultaneous Source Location problem and solve the problem 
exactly on trees. We present a (1 + e) violation of the capacities PTAS for graphs 
of bounded treewidth. On general graphs we find a solution with exact number of 
sources which can exceed the capacities by at most a factor of 0(log^ n log log n) . 
We show a O(logn) factor approximation on the number of sources with no 
violation of the capacities for general directed graphs. We believe that many 
interesting applications of this problem involve graphs of low treewidth; many 
of the connectivity graphs of real networks have been observed to have low tree 
width [8]. 

The main open problem is the approximability of SSL on undirected graphs of 
large treewidth. No constant approximation on the number of sources is known, 
even if we allow constant violation of the capacities. The only lower bound on 
approximability with exact capacities is 1.36067 — e. An approximation factor 
asymptotically better than 2 would be a surprising improvement on existing 
vertex cover results [15]. One can also consider adding costs on the edges and/or 
the vertices. 
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A Dealing with Super-Polynomial Capacities 

Some of our analyses assume that there are only a polynomial number of capac- 
ities. Here we present a method to extend those analyses to when the capacities 
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are super-polynomial, allowing us to satisfy demands while exceeding capacities 
by a 1 -I- e factor in those cases. 

When capacities are super-polynomial, we express the flow and capacity of 
each edge as aF"^ for some value of i (which might be different from edge to edge) 
and some value of a which is between F and F^ . This can be done by rounding 
every flow up to the nearest value. Once this is done, we might no longer have a 
feasible flow. The new capacities (which might be larger than the old capacities 
by a 1 factor) will not be exceeded. However, flow conservation may no 
longer hold. Consider any node. If the original inflow was / then the outflow was 
/ also. But it is possible that after this rounding upwards, the inflow is still / 
and the outflow has increased to f{l + ^). We consider such a flow to be feasible. 
This means that a node might effectively “generate” some amount of flow, but 
this amount is at most -p of the inflow. 

Now consider any graph with a feasible flow under the representation de- 
scribed above. We would like to transform this into a real flow. If we consider 
splitting the “generated” flow equally among outgoing edges, we will see that 
as flow passes through a node the percentage of “real” flow might decrease by a 
factor of -p^. In the worst case, some edge might have a fraction of “real” flow 
equal to (p^)” > 1 - f • 

We let = p. It follows that we can satisfy at least 1 — e of each demand 
while exceeding capacities by a factor of 1 -I- ^ . Since we can scale the flows and 
capacities, this means we can satisfy the demands while exceeding capacities by 
1 -I- e factor. 

This is useful in bounding the running time and approximation factor of 
various techniques for approximating SSL. 
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Abstract. In many economic settings, convex figures on the plane are for sale. 
For example, one might want to sell advertising space on a newspaper page. Self- 
ish agents must be motivated to report their true values for the figures as well as 
to report the true figures. Moreover, an approximation algorithm should be used 
for guaranteeing a reasonable solution for the underlying NP-complete problem. 
We present truthful mechanisms that guarantee a certain fraction of the social 
welfare, as a function of a measure on the geometric diversity of the shapes. We 
give the first approximation algorithm for packing arbitrary weighted compact 
convex figures. We use this algorithm, and variants of existing algorithms, to cre- 
ate polynomial-time truthful mechanisms that approximate the social welfare. We 
show that each mechanism achieves the best approximation over all the mecha- 
nisms of its kind. We also study different models of information and a discrete 
model, where players bid for sets of predefined building blocks. 



1 Introduction 

The intersection between Micro-Economic theory and Computer-Science theory raises 
many new questions. These questions were studied recently by researchers from both 
disciplines (see, e.g., the surveys in [13,6]). A leading example for a problem in this 
intersection is the Combinatorial Auction problem. In a combinatorial auction, a finite 
set of heterogenous items is for sale, and each selfish agent has a valuation for every 
subset of these items. As the auction designers, we try to find an allocation of fhe items 
among fhe agenfs fhaf maximizes the “social welfare” (i.e., a set of disjoint packages 
that maximizes the sum of valuations) or at least to find a good approximation. 

In this paper, we study a variant of combinatorial auctions: e.g., a newspaper wants 
to sell an advertising space on a newspaper page. Agents might have different prefer- 
ences about their desired space: the size of the ad, its location on the page, whether its 
figure is rectangular, square, or elliptic etc. Each agent submits a bid for her favorite 
figure, and we try to find fhe allocation that maximizes the social welfare. The underly- 
ing packing problem is known to be NP-hard, even for the very simple case of packing 
2x2 squares ([15]). Thus, we settle for a computationally-efficient mechanisms that 
approximate the social welfare. 

* The authors are grateful to Noam Nisan for many helpful discussions, and to Ron Lavi for his 
comments on an earlier draft. 

K. Jansen et al. (Eds.): APPROX and RANDOM 2004, LNCS 3122, pp. 27-38, 2004. 
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In our model, the plane (91^) is for sale. Let be a finite set of agents (|A^| = n). 
Each agent has a private non-negative valuation v, G 91^ for a single compact convex^ 
figure Si and for any figure that contains it (other figures have a valuation of 0). Every 
agent submits a bid for her desired figure (e.g., an advertiser might want the lower half 
of the first page in a newspaper). After receiving all bids, the auctioneer determines a 
set of winning agents with disjoint figures and the payment that each agent should pay. 
Note that the agents demand figures in fixed locations in the plane, and the auctioneer 
cannot translate or rotate them. Bidding for convex figures is common in many real- 
life scenarios that involve “geometric” bids, e.g., selling real-estate lots, newspaper ads 
and spectrum licenses in different locations. In most existing real-estate or advertising 
auctions, agents are forced to bid on predefined disjoint figures. This might result in 
inefficient allocations that can be avoided by allowing the agents to bid for arbitrary 
figures (which in turn makes the computational problem harder). 

Note that the problem addressed in this paper is more than just finding an algorithm 
with a good approximation ratio. The agents in our model are selfish, and they may 
report untruthful information if this is beneficial for them. We want to design incentive- 
compatible mechanisms, in which each agent uses a strategy that is best for her own 
selfish interest (a dominant strategy), and yet, a certain approximation for the social 
welfare is guaranteed. A general scheme for achieving a welfare-maximizing incentive- 
compatible mechanism is the family of Vickrey-Clarke-Groves (V CG) mechanisms (see 
[11] for a review). However, for implementing such mechanisms we must allocate the 
goods optimally (otherwise it will not be truthful [14]). Thus, since finding the optimal 
allocation is an AP-hard problem, we must find incentive-compatible mechanisms that 
are non-VCG. Almost all the non-VCG mechanisms currently known are for models 
where the agents hold a single secret value (single-parameter models)^. 

In light of these impossibilities, we assume that each agent is interested in a single 
figure. Lehmann et al. [10] initiated the study of the Single-Minded Bidders model for 
combinatorial auctions. Our model is unique since the bids have some common geomet- 
ric properties (e.g., convexity), and also because we actually auction an infinite (even 
uncountable) number of goods (the points in the plane). 

We differentiate between two models of information (similar to the differentiation 
done in [12,3]) . In the first model, the auctioneer knows which figure each agent wants, 
but does not know how much this agent is willing to pay for this figure. This model is 
the Known Single-Minded (KSM) model, and this is indeed a “single-parameter” model. 
In the second model, called the Unknown Single-Minded (USM) model, both the fig- 
ures and the values are unknown. In the KSM model we should motivate the agents to 
truthfully declare their true values, where in the USM model the agents might submit 
untruthful bids both for their desired figures and their values^. 



^ Actually we prove our results for a more general model that allows non-convex bids as well, 
with some more general restrictions. 

^ Recent results ([9]) show that in many reasonable settings, essentially no IC mechanisms exists 
for multi-parameter models, except the family of weighted VCG mechanisms. 

^ Note that the USM model does not fit into the “single-parameter” definition, since the agents 
have both their values and their figures as their secret data. 




Computationally-Feasible Truthful Auctions for Convex Bundles 29 



Another differentiation we make is between a continuous and a discrete model. In 
the continuous model, each agent is interested in an arbitrary compact convex figure in 
91^. For example, if a piece of land is for sale, each agent can draw an arbitrary figure on 
the map. In the discrete model, the plane contains predefined atomic building blocks (or 
tiles), and each agent is restricted to bids for a set of building blocks which are contained 
in some convex figure. For example, if we wanted to resell the island of Manhattan, the 
basic building blocks would be street blocks bounded between two consecutive streets 
and two consecutive avenues. These blocks are typically convex, though not necessarily 
rectangular (e.g., because of the diagonal Broadway Avenue). 

Related Work: 

Our research relates to a sub-field of Micro-Economics called Mechanism Design, 
which studies ways to design mechanisms that encourage agents to behave in a way 
that results in some desired global properties (see, e.g., [11]). Nisan and Ronen [14] 
introduced this concept to CS by the name Algorithmic Mechanism Design. 

Weighted packing of rectangles in the plane was studied in several papers. 
Hochbaum and Maass [7] proposed a shifting strategy for a special case of square pack- 
ing, and generalizations for arbitrary squares appear in, e.g., [5,4]. Khanna et al. [8] 
used similar methods in a model where axis-parallel rectangles lie in a « x n grid. 
They presented an algorithm that runs in polynomial time and achieves an 0{log{n))- 
approximation for the optimal welfare. However, it is an open question whether a better 
approximation (and in particular, a constant approximation) exists for this problem. 

Our Contribution: 

We measure the quality of the approximations achieved by mechanisms in our model 
according to an aspect ratio R, which measures how diverse are the dimensions of 
the figures demanded by the agents. R is defined as the ratio between the maximal 
diameter of a figure and the minimal width of a figure (formally defined in Section 2). 
For different families of figures, we construct IC mechanisms, either for the KSM or for 
the USM models. This mechanisms are also individually- rational, i.e., agents will not 
pay more than they value the figure they receive (if any). Therefore, our approximation 
improves as the dimensions of the figures become closer^. 

We study three different families of figures: compact convex figures, rectangles, and 

4 

axis-parallel rectangles. For convex figures in the USM model, we achieve an 0{R^ )- 
approximation to the social welfare. If the bids are restricted to rectangles (not neces- 
sarily axis-parallel), we achieve a better approximation of 0{R). 

If the agents bid for axis-parallel rectangles, we can use a slight modification of the 
algorithm due to Khanna et al. [8] to design an IC mechanism that achieves an approx- 
imation ratio of 0{log{R)) (the best known approximation ratio for this problem). 

We also present a novel allocation algorithm that achieves an 0(R)-approximation 
for packing arbitrary compact convex figures, and as far as we know this is the first 
approximation algorithm for this problem. We use this algorithm for constructing an IC 
mechanism, with the same approximation ratio, for the KSM model. 

For instance, when all figures are disks with the same radius up to a constant, our mechanisms 
achieve a constant approximation. 
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The incentive-compatible mechanisms we present for the USM model are based on 
a family of greedy algorithms presented by Lehmann et al. [10]. For standard combina- 
torial auctions, Lehmann et al. normalized the values by |5|“, where j^j is the number of 
items in the bundle S and a is some real constant, and then run a simple greedy alloca- 
tion algorithm on the normalized values. They showed that choosing oc = 5 guarantees 
the best polynomial approximation ratio (unless NP = ZPP). 

We present mechanisms, called a-greedy mechanisms, that normalize the values 
using the geometric area of the figures. That is, for a G 91 we assign a normalized 
value of ^ to a bid with a value v for a figure with a geometric area q. We show 
that, somewhat surprisingly, for compact convex figures the optimal value for a is 

4 

resulting an (9(7? 3) -approximation. The difference between the results of Lehmann et 
al. and ours derives from the different divisibility properties of packages in the two 
models. In their model, a finite set of goods is traded, and for a package to intersect 
“many” disjoint packages, its size must be “large”. Flowever, in our continuous model, 
a small package can intersect many disjoint packages. 

4 

For our discrete model, we present a mechanism that achieves an 0(Rt ) approx- 
imation. However, if the ratio between the minimal width of a figure and the sizes 
of the building blocks (we denote by Q) is smaller than the aspect ratio R, we can 

achieve a better approximation of 0{R • 2“ ), by running the a-greedy algorithm with 

* _ log(R) 5 
- 2log{R)+log{Q) ■ 

The paper’s organization: Section 2 describes our model. Section 3 describes our 
results for the USM model, both for the continuous case and the discrete case. Section 
4 presents the results for the KSM model, and Section 5 concludes with a discussion of 
future work. All proofs are given in the full version of the paper ([2]). 



2 Model 

Let B denote the family (set) of bids (figure-value pairs) of fhe agenfs®, fhaf is B — 
{(s,, v,)|f G N}. Let F denote the family of agents figures, that is F = {5,jf G A^}. Given 
a family of bids B, we aim to maximize the social welfare, i.e., find a collection of non- 
conflicting bids (bids for disjoint figures) that maximizes the sum of valuations. For a 
subset C C Af of agents with disjoint figures, denote the value of C by V{C) = XiecL- 
We denote the set of disjoint figures fhaf achieves the maximal welfare by OPT (to 
simplify the notation we assume that there are no ties, so there is a single optimal 
solution), i.e., 

V{OPT) = max V{C) 

CCN\ Vijec Sinsj=& 



^ Thus, if one can embed the goods of a traditional combinatorial auction as building-blocks in 
the plane, such that each agent bids for building-blocks contained in some convex figure, then 
our approximation scheme improves the approximation ratio achieved in [10]. 

® Since all the mechanisms we consider are truthful, we use the same notation for the secret 
information and the declared information (bid), except of the IC proofs. 
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Definition 1. A mechanism consists of a pair of functions (G,P) where: 

- G is an allocation scheme (rule) that assigns a figure in T (where T is the set 
of compact convex figures in the plane) to every agent such that the figures are 
disjoint, i.e. G(B) C and for every i j in N, Gj(B) H GfiB) — 0 (where we 
denote the figure received by agent i by GfB) ). 

- P is a payment scheme, i.e.foranyB,P(B) Denote the payment paid by agent 

ibyPfB). 

All allocation rules we present in the paper, allocate to an agent either her requested 
figure or the empty figure. All the payment rules we consider are normalized, that is, 
a losing agent pays zero. Additionally, each agent pays a non-negative payment. We 
assume quasi-linear utilities and that the agents have no externalities (the utility for 
each agent does not depend on the packages received by the other agents), i.e., the 
utility of each agent i is ufB) = vfiGfB)) —PfiB). The agents are rational, so each 
agent chooses a bid that maximizes her own utility. 

A mechanism is incentive-compatible (IC) if declaring their true secret information 
is a dominant strategy for all the agents. In the KSM model, it means that for any 
set of values reported by the other agents, each agent cannot achieve a higher utility 
by reporting an untruthful value, i.e., Vi V 5 -,- Vv' > Ui((si,v[),B^i), 

where denote the family of all bids except i’s bid . In the USM model, IC means that 
each agent’s best strategy is to report both her figure and her value truthfully, regardless 
of the other agents’ reports, i.e., Vi VB_,- Vv;,i' Ui({si,Vi),B^j) > Ui{{s'i,v\),B^i). 

An incentive-compatible mechanism is also individually rational (IR) if for any 
agent i, bidding truthfully ensures him a non-negative utility. That is, Vi VB-, 

Ui((si,Vi),B-i)>f). 



Geometric Definitions: 

We state our approximation bounds as functions of few geometric properties of the 
family of hgures the agents bid for. We use standard definitions of diameter and width 
of compact figures in 91 ^ : 

The diameter d^ of a compact set z is the maximal distance between any two points 
in the set, i.e. d^ = maXpj_p2Gz ||fi — P2II2 (||fi — /’2II2 is the Euclidean distance between 
pi and p 2 ). The width w^ of a compact set z is the minimal distance between the closest 
pair of parallel lines such that the convex set z lies between them. 

Definition 2. Given a family of figures F in 91^, the maximal diameter L is the maximal 
diameter of a figure in F, and the minimal width W is the minimal width of a figure in 
F. The aspect ratio R is the ratio between the maximal diameter and the minimal width. 
That is, L = maXjgf d^ , W = min^gf w^ , R — 

The aspect ratio describes how diverse is the family of figures with respect to the fig- 
ures’ diameter and width^. The approximations our mechanisms achieve are asymptotic 
functions of the aspect ratio R. 

^ For example, if all the figures are disks with the same radius, then R= 1. If we have disks of 
diameter 10 and 5x2 rectangles, then R = ^ = 5. 
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Denote the geometric area of a compact figure z by q{z). We assume that the di- 
ameter, width and area of any agent’s figure are polynomial-time computable. We also 
assume that given any two agent’s figures, we can decide if the two figures are disjoint 
in polynomial time*. 

The Discrete Model: 

In the discrete model, there is a set of atomic building blocks (we call tiles) embedded 
in the plane. Each agent desires a bundle of tiles that are exactly the ones that are fully 
contained in some compact convex figure, and she reports this figure and her value 
for the set of tiles contained in it. We assume that all tiles have similar dimensions 
(specifically, each tile contains a disk of some positive diameter Wo and its diameter is 
at most 2Wq). Two agents are non conflicting if there is no tile which is fully contained 
in the two figures they report. 

For a given family of bids, we define the width-ratio Q = ^. The width-ratio gives 
an upper bound on the width of any figure, with respect to the size of the tiles. Clearly, 
we can assume that Q > 1. 



The Greedy Mechanism: 

Lehmann et al. [10] presented the following family of greedy mechanisms for combi- 
natorial auctions: 

Given a family of bids and some function / on the figures, such that / assigns a 
positive real value to any non empty figure (w.l.o.g. all figures are non empty), the 
greedy allocation algorithm picks an allocation ALG, by the following scheme: 

Create a list of the bids sorted from high to low according to their values normalized 
byf(i.e.,^>^>...>^). 

While the list is not empty, choose a figure s, for which the normalized value is 
highest in the remaining list (with a consistent tie breaking). Add i to the allocation 
ALG and update the list by removing all bids for figures that intersect Si. 

The specific algorithm is determined by the choice of the function (or norm) /. 
Lehmann et al. suggested using the norm for combinatorial auctions, where |j| 
is the size of the package s and a is some non-negative constant. We generalize this 
method for compact figures in 91^ and define the a-greedy algorithm to use the norm 
where q{z) is the area of figure z 



Definition 3. The (X-greedy mechanism is a mechanism which uses the Ct-greedy algo- 
rithm as its allocation scheme, where a winning agent i pays according to the following 
payment scheme: 

Let j be the first agent to win, among all the agents whose figures intersects agent 

q{s 'V ' 

i’s figure, when the greedy algorithm runs without i. If such j exists, i pays 
otherwise i pays 0. Losing agents pay 0. 



The properties of the a-greedy mechanisms, proved in [10], also hold in our model: 

* Note that for polygons the above assumptions hold, and that any compact convex figure can 
be approximated (as good as one wants) by a polygon. 

^ For example, the 0-greedy algorithm sorts the figures according to their values and the 1- 
greedy algorithm sorts the figures according to their value per unit of area. 
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Theorem 4. (essentially due to [10]) For every a, the (X- greedy mechanism is polyno- 
mial time, individually rational and incentive compatible for agents bidding for com- 
pact figures in the USM model. 



3 The Unknown Single-Minded Model 

This section considers the problem of designing a polynomial-time, individually- 
rational and incentive-compatible mechanisms, which guarantee some fraction of the 
social efficiency for the USM model. We study three families of hgures: convex hgures 
and rectangles in the continuous model, and convex figures in the discrete model. We 
use the a-greedy mechanism to create mechanisms with the desired properties for the 
three families. For each family, we hnd the value of a that optimizes the asymptotic 
approximation for the social welfare, over all a-greedy mechanisms. For convex figures 
in the continuous model, we show that a = ^ achieves an 0(R ^ ) -approximation for the 
social welfare. For rectangles in the continuous model, we improve the above result and 
show that a = 5 achieves an (?(/?) -approximation for the social welfare. Finally, for 
convex hgnres in the discrete model, we show that a careful choise of a as a function 

4 

of R and Q, gives an approximation ratio between O(R^) and 0(R). The proofs of all 
above results are based on a single general result presented in the fnll version [2]. 

3.1 Convex Figures and Rectangles in the Continuous Model 

The following lemma presents a lower bound on the approximation ratio that can be 
achieved by a-greedy mechanisms, by presenting two constrnctions that are in a sense 
the hardest inputs for these mechanisms. 

Lemma 5. The a-greedy mechanism for compact convex figures achieves an 
Q(/;2(l-a)) approximation for any a < 1, and an Q,(R^^^)-approximation for any a > 

4 

0. Therefore, the a-greedy mechanism for compact convex figures achieves an Q,(R^ )- 
approximation. 

Proof sketch: Each of the lower bounds is achieved by a construction that can be built 
for any R large enough. The left part of Figure 1 illustrates the construction nsed to 
prove the bonnd and the right part is used to prove the bound. In 

the left example, a large rectangle contains Q{R?) small disjoint squares with a side 
of length W . On the right example, a small disk intersects Q(R) disjoint triangles, with 
equal area of Q(WL). In both constructions, there is one figure z (hlled by small vertical 
lines) that is chosen by the greedy mechanism, while the socially optimal mechanism 
chooses a family of disjoint hgnres (small horizontal lines), each intersects z. The value 
of z is chosen such that its normalized value is a bit greater than 1 , and the rest of 
the hgures have a normalized value of 1 . The value for a that minimizes the worst case 
approximation is therefore yielding an Q.(R ^ ) lower bound. □ 

Next, we show that the ^-greedy mechanism achieves an (9 (7?^ (-approximation 
(which, by the above lemma, is the best over all the a-greedy mechanisms). To prove 
this result, we use few elementary geometric properties of convex hgures. First, for any 
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Fig. 1. Approximation bounds for the a-greedy mechanism 



compact convex figure z, q{z) = Q{d{w^). Additionally, the perimiter of z (denoted by 
is contiguous, and = Q{d^) (the constants in both cases are independent of the 
figure z). These properties are sufficient for the approximation to hold. 

Theorem 6. When agents bid for compact convex figures in the plane with an aspect 
ratio R, the ^ - greedy mechanism achieves an 0{R'^)-approximation, and this is the 
best asymptotic approximation achievable by an (X-greedy mechanism (for any a). This 
mechanism is individually rational and incentive compatible for the USM model, and it 
runs in polynomial time. 

Proof sketch: By Theorem 4 the mechanism is IR, IC and it runs in polynomial time. 

4 

Next, we present the idea behind the proof of the (9 (R^) -approximation ratio. By the 
definition of the a-greedy algorithm, any figure is either a winner or intersects a winner. 
It is the hardest to prove the approximation bound if the set of winners in the optimal 
solution OPT is disjoint to the set of winners ALG picked by the a-greedy algorithm. 
We map each agent x G OPT to a winner z G ALG that intersects x. We then bound the 
sum of values of any disjoint set of agents’ figures that intersects z, by partitioning them 
to figures that are contained in z and to figures that are not. We use Holder inequality 
and some simple geometric properties of convex figures to show that the upper bounds 
for a = 5 , for both the contained figures and the rest of the intersecting figures, match 
the lower bounds presented in Lemma 5. □ 

If agents are only interested in rectangles (not necessarily axis parallel), than we 
can derive a stronger result of an O(R) -approximation for the social welfare. While 
the construction of a large rectangle containing Q.(R^) small rectangles (as presented in 
Lemma 5) is still possible, the second construction is not. For rectangles, it is impossible 
for a small rectangle to hit (intersect but not contain) many disjoint rectangles. 

Theorem 7. When agents bid for rectangles in the plane with an aspect ratio R, the | 
- greedy mechanism achieves an 0(R) -approximation for the social welfare, and this is 
the best asymptotic approximation achievable by an (X-greedy mechanism (for any a)^*^. 
This mechanism is individually rational and incentive compatible for the USM model, 
and it runs in polynomial time. 

We actually prove a stronger statement. We show that f2(R)-approximation is the best over all 
the greedy mechanisms that sort the bids according to some function of the value and the area 
only (specifically, this includes the function 
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3.2 Convex Figures in the Discrete Model 

We now turn to look at the discrete model. We first define fhe mechanism we use for 
the discrete model, we then present the mechanism properties 



Definition 8. The Discrete Model Greedy Mechanism is a mechanism that given bids 
for compact convex figures in 91^ in the discrete model, does the following: IfQ > R then 
it runs the ^-greedy mechanism, and if Q <R then it runs the a* - greedy mechanism 



for a* 



logiM 

2log{R)+log(Q) ■ 



Theorem 9. Consider that the agents bid for compact convex figures in 91^ in the dis- 
crete model, with an aspect-ratio R and a width-ratio Q. Then, the Discrete Model 

4 

Greedy Mechanism achieves an 0{R^)-approximation for the social welfare. More- 
over, when Q < R it achieves a better approximation of 0{R ■ ). This mechanism 

achieves the best asymptotic approximation among all the mechanisms that choose a 
as a function ofR and Q, and in particular it is asymptotically better than the (X-greedy 
mechanism for any a. Additionally, the mechanism is IR and IC for the USM model, 
and it runs in polynomial time. 



4 The Known Single-Minded Model 

In this section, we present polynomial-time mechanisms for different families of figures 
in the Known Single-Minded (KSM) model (where the auctioneer knows the desired 
figure of each agent, hut does not know her value for the figure). We start by presenting 
an auction for general compact convex figures. We achieve an 0(/?)-approximation for 

4 

the social welfare, which is better than the )-approximation that we proved for the 
USM model. Next, we present a mechanism (based on an algorithm from [8]), which 
gives an (9 ( fog (/?)) -approximation for axis-parallel rectangles^^. 

4.1 Mechanisms for Convex Figures 

Consider the mechanism called the “Classes-by-Area 1-Greedy Mechanism” ( CBA-IG 
mechanism) presented in Figure 2. This mechanism divides the bids of the agents to 
classes according to the figures’ geometric area, runs a 1 -greedy algorithm in each class, 
and allocates the figures fo agents in the class that achieved the highest result. From the 
algorithmic aspect, this is the first algorithm for packing weighted convex figures fhaf 
we know of, and it achieves an 0(l?)-approximation. We use this algorithm to construct 
a polynomial-time and IC mechanism with the same approximation ratio for the social 
welfare. The payments are exactly the “critical-values” for the agents, i.e., the minimal 
declaration for which they still win the auction. 

* ^ Note that an agent can manipulate the values of a by affecting R and Q. Therefore, for incentive 
compatibility, the mechanism is assumed to know the true values of R and Q. 

This approximation is exponentially better than the approximation ratio we achieve for this 
problem in the USM model and than the ratio we achieve for general convex figures in the 
KSM model. 
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The Classes-by-Area 1-Greedy (CBA-IG) Mechanism: 

Allocation '. 

Step 1: Divide the given input to m = 2log(R) classes according to their area. 

A hgure s belongs to class c if q{s) e [W^ • • 2'^+*) (for c G {0, . . . ,m — 1}). 

Step 2: Perform the 1-greedy algorithm per each class. Denote the welfare achieved 
by class c by 

Step 3: Output the allocation in the class c for which the 1-greedy algorithm achieved 
the highest welfare, i.e., c G argmax^^sfi 

Payments'. 

Denote the winning class as class 1, and the class with the second-highest welfare as 
class 2. Let V}^- = V^ — v;, and let j be the first figure that intersects figure i and wins, 
when we run the greedy algorithm where agent i is removed. Let Zi be if 
such j exists, and 0 otherwise. 

A winning agent i pays: P(i) = max{V^ — VA , Zj}, and any losing agent pays 0. 



Fig. 2. A mechanism for selling arbitrary convex figures. This mechanism is incentive compatible 
in the KSM model and achieves an 0(f?)-approximation for the social welfare 



The payments in the CBA-IG mechanism are chosen as follows: to win the auction, 
each agent should be both a winner in her class and her class should beat all other 
classes. Bidding above the value z, in the mechanism’s description, is a necessary and 
sufficient condition for agent i to win in her class. However, if agent i bids below — 
y*, and still wins in her class, her class will definitely lose. 

Theorem 10. When the agents bid for compact convex figures in 91^ with an aspect 
ratio R, the CBA-IG mechanism achieves an 0{R)-approximation. This mechanism is 
IR and IC for the KSM model^^, and runs in polynomial time. 

Proof sketch: We show that the approximation ratio achieved in each class is 0{Rc), 
where Rc is the aspect ratio in class c. Due to a general proposition we prove, the ap- 
proximation ratio achieved by choosing the best class is 0{fl^Rc). Finally, we show that 
= 0{R), by dividing this sum to two geometric series. For proving IC, we show 
that the given payments are indeed the critical values for the agents, i.e. the smallest 
declarations for which they still win. □ 

4.2 Mechanisms for Axis-Aligned Rectangles 

In the full version of this paper ([2]) we present an allocation algorithm, called the 
Shifting Algorithm, which is based on an algorithm by Khanna et al.([8]) with some mi- 
nor changes. They studied a model where axis-aligned rectangles lie in an n x n array, 
and they proved an G(/og(n))-approximation for the weighted packing problem. This 
approximation is the best approximation currently known for weighted packing of axis- 
parallel rectangles. Our algorithm gives an 0{log{R) ) -approximation in a slightly more 
general model where the rectangles can lie in any axis-parallel location in the plane. By 

We observe that this mechanism is not IC in the USM model. 

I.e. the ratio between the maximal diameter and the minimal width of figures in this class. 
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carefully defining a payment scheme, we use this allocation rule for designing an IC 
polynomial-time mechanism achieving an (9(/og(/?))-approximation for the social wel- 
fare. We call this mechanism the Shifting Mechanism and we summarize its properties 
in the following theorem'^: 

Theorem 11. When the agents bid for axis-parallel rectangles in 91^ with an aspect 
ratio R, the Shifting Mechanism achieves an 0{log{R))-approximation. This mecha- 
nism is individually rational and incentive compatible for the KSM model, and runs in 
polynomial time. 

5 Conclusion and Further Research 

In this paper, we study auctions in which the agents bid for convex hgures in the plane. 
We present mechanisms that run in polynomial time, in which the selhsh players’ best 
strategy is to send their true private data to the auctioneer. We suggest using the aspect 
ratio R, which measures how diverse are the dimensions of the hgures, for analyzing 
the economic efficiency of the mechanisms. 

In the KSM model, we were able to achieve the best approximation currently known 
for weighted axis-parallel rectangle packing (log{R)) in an 1C mechanism. Lehmann et 
al. [10] showed that the best polynomial-time approximation for combinatorial auctions 
(for single minded bidders) can be achieved with an IC mechanism. On the other hand, 
recent results showed settings in which the optimal algorithmic approximation ratio 
cannot be achieved by IC mechanisms (see, e.g., [1,9]). Whether such gap exists in our 
model is an interesting open question: 

Open Problem: Can the best polynomial-time approximation schemes for packing 
convex figures (general figures, rectangles, or axis-parallel rectangles) be implemented 
by incentive-compatible mechanisms? 

The mechanism for combinatorial auctions presented in [10] achieves the same ap- 
proximation both for the USM model and the KSM model. Our results might indicate 
that, in our model, a gap exists between the approximation achievable in both informa- 
tion models. For general convex figures, the approximation we achieve in the KSM and 

4 

the USM models are 0(R) and O(R^), respectively. For axis-parallel rectangles, the 
gap in our results is even exponential. 

Open Problem: In settings where the agents are “single minded’’, is there a gap be- 
tween the best approximation achievable in the KSM and in the USM models? 

We present some novel algorithmic results regarding packing of convex hgures and 
arbitrary rectangles. We have not been able to show that these results are tight. 

Open Problem: Is there an o(R) -approximation scheme for packing general convex 
figures, or even for packing rectangles (not necessarily axis-parallel)? 

Our results may also be useful in deriving approximation results for the problem of 
packing weighted convex bodies in dimensions higher than two^®. 

An easy observation is that the Shifting Mechanism is not IC in the USM model. We also 
note that the Shifting Mechanism achieves an ^(/^[-approximation for general rectangles (not 
necessarily axis-parallel). 

However, the economic interpretation of such auctions is not always clear. 
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Abstract. In this paper we investigate the computational complexities 
of a combinatorial problem that arises in the reverse engineering of pro- 
tein and gene networks. Our contributions are as follows: 

— We abstract a combinatorial version of the problem and observe 
that this is “equivalent” to the set multicover problem when the 
“coverage” factor fe is a function of the number of elements n of the 
universe. An important special case for our application is the case 
in which k = n — 1. 

— We observe that the standard greedy algorithm produces an approx- 
imation ratio of l7(logn) even if k is “large” i.e. k — n — c for some 
constant c > 0. 

— Let 1 < a < n denotes the maximum number of elements in any 
given set in our set multicover problem. Then, we show that a non- 
trivial analysis of a simple randomized polynomial-time approxima- 
tion algorithm for this problem yields an expected approximation 
ratio E[r(o, fc)] that is an increasing function of a/k. The behavior 
of E[r(a, fe)] is “roughly” as follows: it is about ln(a/fc) when a/k is 
at least about « 7.39, and for smaller values of a/k it decreases to- 
wards 2 exponentially with increasing k with \ima/k^o E[r(a, k)] < 2. 
Our randomized algorithm is a cascade of a deterministic and a ran- 
domized rounding step parameterized by a quantity /3 followed by a 
greedy solution for the remaining problem. 



1 Introduction 

Let [x, y] is the set {x, x-l- 1, x-|-2, . . . , y} for integers x and y. The set multicover 
problem is a well-known combinatorial problem that can be defined as follows. 
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Problem Name: SCfc. 

Instance <n,m,k >: An universe U = [1, n], sets Si,S 2 , - ■ ■ , Sm C U 
with = U and a “coverage factor” (positive integer) k. 

Valid Solutions: A subset of indices / C [l,m] such that, for every 
element x G U, \j G I : x G Sj \ > k. 

Objective: Minimize |/|. 

SCi is simply called the Set Cover problem and denoted by SC; we will 
denote an instance of SC simply by <n,m> instead of <n,m,l> . 

Both SC and SCfc are already well-known in the realm of design and analysis 
of combinatorial algorithms {e.g., see [14]). Let 3 < a < n denote the maximum 
number of elements in any set,z.e., a = max^gji miijiSil}. We summarize some of 
the known relevant results for them below. 

Fact 1 

(a) [4] Assuming NP ^ instances < n,m > of the SC 

problem cannot he approximated to within a factor o/(l — e) Inn for any constant 
0 < e < 1 in polynomial time. 

(b) [14] An instance <n,m,k> of the SCfc problem can he 

{1+ln a) -approximated in 0{nmk) time by a simple greedy heuristic that, at every 
step, selects a new set that covers the maximum number of those elements that 
has not been covered at least k times yet. It is also possible to design randomized 
approximation algorithms with similar expected approximation ratios. 

1.1 Summary of Results 

The combinatorial problems investigated in this paper that arise out of reverse 
engineering of gene and protein networks can be shown to be equivalent to SCfc 
when fc is a function of n. One case that is of significant interest is when k is 
“large”, z.e., k = n — c for some constant c > 0, but the case of non-constant c is 
also interesting (cf. Questions (Ql) and (Q2) in Section 2). Our contributions 
in this paper are as follows: 

— In Section 2 we discuss the combinatorial problems (Questions (Ql) and 
(Q2)) with their biological motivations that are of relevance to the reverse 
engineering of protein and gene networks. We then observe, in Section 2.3, 
using a standard duality that these problems are indeed equivalent to SCfc 
for appropriate values of k. 

— In Lemma 1 in Section 3.1, we observe that the standard greedy algorithm 
SCfc produces an approximation ratio of l7(logn) even if k is “large”, i.e. 
k = n — c for some constant c > 0. 

— Let 1 < a < n denotes the maximum number of elements in any given set 
in our set multicover problem. In Theorem 2 in Section 3.2, we show that a 
non-trivial analysis of a simple randomized polynomial-time approximation 
algorithm for this problem yields an expected approximation ratio E[r(a, k)] 
that is an increasing function of a/k. The behavior of E[r(a, k)] is “roughly” 
as follows: it is about ln(a/fc) when a/k is at least about « 7.39, and for 
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smaller values of a/k it decreases towards 2 exponentially with increasing k 
with lim^/fc^o E[r(a, k)] < 2. More precisely, E[r(a, fc)] is at most 

1 + Ina, if fc = 1 

(l + ln(a/ (fc — 1)), if a/(fc — 1) > and fc > 1 

min{ 2 + 2 • 2 + (e"2 + e-9/») • f } 

« min{ 2 + 2 • 2 + 0.46 • | } if a/(fc — 1) < and fc > 1 

Some proofs are omitted due to lack of space. 

1.2 Summary of Analysis Techniques 

— To prove Lemma 1, we generalize the approach in Johnson’s paper [6]. A 
straightforward replication of the sets will not work because of the depen- 
dence of k on n, but allowing the “misleading” sets to be somewhat larger 
than the “correct” sets allows a similar approach to go through at the ex- 
pense of a diminished constant. 

— Our randomized algorithm in Theorem 2 is a cascade of a deterministic and 
a randomized rounding step parameterized by a quantity f3 followed by a 
greedy solution for the remaining problem. 

— Our analysis of the randomized algorithm in Theorem 2 uses an amortized 
analysis of the interaction between the deterministic and randomized round- 
ing steps with the greedy step. For tight analysis, we found that the standard 
Chernoff bounds such as in [1, 2, 10, 14] were not always sufficient and hence 
we had to devise more appropriate bounds for certain parameter ranges. 



2 Motivations 

In this section is to define a computational problem that arises in the context of 
experimental design for reverse engineering of protein and gene networks. We will 
first pose the problem in linear algebra terms, and then recast it as a combina- 
torial question. After that, we will discuss its motivations from systems biology. 
Finally, we will provide a precise definition of the combinatorial problems and 
point out its equivalence to the set multicover problem via a standard duality. 

Our problem is described in terms of two matrices A G and B G 
such that: 

— A is unknown] 

— B is initially unknown, but each of its columns, denoted as B\, B 2 , ■ ■ ■ , Bm, 
can be retrieved with a unit-cost query, 

— the columns of B are in general position, i.e., each subset oi k <n columns 
of B is linearly independent, 

— the zero structure of the matrix C = AB = (cy) is known, i.e., a binary 
matrix C° = (c°^) G {0, 1}"^'" is given, and it is known that Cy = 0 for 
each i,j for which cF =0. 
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The objective, “roughly speaking” , is to obtain as much information as possible 
about A (which, in the motivating application, describes regulatory interactions 
among genes and/or proteins), while performing “few” queries (each of which 
may represent the measuring of a complete pattern of gene expression, done 
under a different set of experimental conditions) . 

Notice that there are intrinsic limits to what can be accomplished: if we 
multiply each row of A by some nonzero number, then the zero structure of C is 
unchanged. Thus, the best that we can hope for is to identify the rows of A up 
to scalings (in abstract mathematical terms, as elements of the projective space 
pn-i)^ To better understand these geometric constraints, let us reformulate the 
problem as follows. Let Ai denote the row of A. Then the specification of 
amounts to the specification of orthogonality relations Ai ■ Bj = 0 for each 
pair i,j for which c? = 0. Suppose that we decide to query the columns of B 
indexed by J = {ji, . . . ,ji} . Then, the information obtained about A may be 
summarized as Ai G Hj^, where “T” indicates orthogonal complement, and 

Hjg = span {Bj,j G Ji} , 

Jr = {j I / G J and c°- = 0} . (1) 

Suppose now that the set of indices of selected queries J has the property: 

each set Ji, i = 1, . . . , n, has cardinality > n — k, (2) 

for some given integer k. Then, because of the general position assumption, the 
space Hjg has dimension > n — k, and hence the space j ^ has dimension at 
most k. 

The most desirable special case is that in which k = 1. Then dimTij^ < 1, 
hence each Ai is uniquely determined up to a scalar multiple, which is the best 
that could be theoretically achieved. Often, in fact, finding the sign pattern 
(such as “(+,+,—,0,0,—,...)”) for each row of A is the main experimental 
goal (this would correspond, in our motivating application, to determining if 
the regulatory interactions affecting each given gene or protein are inhibitory or 
catalytic). Assuming that the degenerate case Hji = {0} does not hold (which 
would determine Ai = 0), once that an arbitrary nonzero element v in the 
line j j has been picked, there are only two sign patterns possible for Ai (the 
pattern of v and that of —v). If, in addition, one knows at least one nonzero sign 
in Ai, then the sign structure of the whole row has been uniquely determined 
(in the motivating biological question, typically one such sign is indeed known; 
for example, the diagonal elements an, i.e. the zth element of each Ai, is known 
to be negative, as it represents a degradation rate). Thus, we will be interested 
in this question: 

find J of minimal cardinality such that | Jj| > n — 1, z = 1, . . . , rz. (Ql) 

If queries have variable unit costs (different experiments have a different associ- 
ated cost), this problem must be modified to that of minimizing a suitable linear 
combination of costs, instead of the number of queries. 
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More generally, suppose that the queries that we performed satisfy (2), with 
k > 1 but small k. It is not true anymore that there are only two possible sign 
patterns for any given Ai, but the number of possibilities is still very small. For 
simplicity, let us assume that we know that no entry of Ai is zero (if this is 
not the case, the number of possibilities may increase, but the argument is very 
similar). We wish to prove that the possible number of signs is much smaller 
than 2". Indeed, suppose that the queries have been performed, and that we 
then calculate, based on the obtained Bj's, a basis {fi, . . . , Vk\ of j ^ (assume 
dimTij^ = k; otherwise pick a smaller k). Thus, the vector Ai is known to 

k 

have the form X^Vr for some (unknown) real numbers Ai, . . . , Afe. We may 

r—1 

assume that Ai yf 0 (since, if Ai = vector evi + X)r =2 

with small enough e, has the same sign pattern as Ai, and we are counting 
the possible sign patterns). If Ai > 0, we may divide by Ai and simply count 
how many sign patterns there are when Ai = 1; we then double this estimate 
to include the case Ai < 0. Let Vr = col (vi^., . . . , n„r), for each r = l,...,fc. 
Since no coordinate of Ai is zero, we know that Ai belongs to the set C = 
Ln^ where, for each 1 < s < n, is the hyperplane in ^ 



consisting of all those vectors (A 2 ,...,Afe) such that X)r =2 
each connected component of C, signs patterns are constant. Thus the possible 
number of sign patterns is upper bounded by the maximum possible number of 
connected regions determined by n hyperplanes in dimension A: — 1. A result of L. 
Schlafli (see [3, 11], and also [12] for a discussion, proof, and relations to Vapnik- 
Chervonenkis dimension) states that this number is bounded above by <P{n, k — 
1), provided that k — l<n, where <P{n, d) is the number of possible subsets of an 

__ , ,, . ,.d 

n-element set with at most d elements, that is, <P{n, d) = 



d 

E 

z=0 



< 



2 ^ 

d\ 



< 



\~d) 



. Doubling 



the estimate to include Ai < 0, we have the upper bound 



2<P{n, k—1). For example, <P{n, 0) = 1, <P{n, 1) = n+1, and <P{n, 2) = i(n^+n+2). 
Thus we have an estimate of 2 sign patterns when k = 1 (as obtained earlier), 
2n + 2 when k = 2, + n + 2 when fc = 3, and so forth. In general, the number 

grows only polynomially in n (for fixed k). 

These considerations lead us to formulating the generalized problem, for each 
fixed k: find J of minimal cardinality such that \Ji\ > n — k for alH = 1, . . . , n. 
Recalling the definition (1) of Ji, we see that Ji = J f]Ti, where Tj = {j \ c°- = 
0}. Thus, we can reformulate our question purely combinatorially, as a more 
general version of Question (Ql) as follows. Given sets 



Tj C {!,..., m}, z=l,...,n. 



and an integer k < n, the problem is: 

find J C {!,..., to} of minimal cardinality such that j J f} Tij > n — k, 

1 < i < n. (Q2) 
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For example, suppose that k = 1, and pick the matrix G {0, 1}”^” in such a 
way that the columns of C° are the binary vectors representing all the (n— 1)- 
element subsets of n} (so m = n); in this case, the set J must equal 

to} and hence has cardinality n. On the other hand, also with k = 1, 
if we pick the matrix in such a way that the columns of are the binary 
vectors representing all the 2-element subsets of {1, , n| (so to = n{n — l)/2), 
then J must again be the set of all columns (because, since there are only two 
zeros in each column, there can only be a total of 2£ zeros, £ = |J|, in the 
submatrix indexed by J, but we also have that 2£ > n(n — 1), since each of the 
n rows must have > n — 1 zeros); thus in this case the minimal cardinality is 
n{n — l)/2. 

2.1 Motivations from Systems Biology 

This problem was motivated by the setup for reverse-engineering of protein and 
gene networks described in [8,9] and reviewed in [13]. We assume that the time 
evolution of a vector of state variables x{t) = (xi(t), . . . ,Xn{t)) is described by 
a system of differential equations: 

~ 1 ^njPl J • • • J Pm) 

X2 = f2{xi,. . ,,Xn,Pl, . . . ,Pm) 

Xn — fni^Xl^ . . . , Xnj Plj • • • j Pm} 

(in vector form, “x = f{x,p)”), where p = {pi, . . . ,Pm) is a vector of parame- 
ters, representing for instance the concentrations of certain enzymes which are 
maintained at a constant value during a particular experiment. There is a ref- 
erence value p of p, which represents “wild type” (that is, normal) conditions, 
and a corresponding steady state x. That is, f{x,p) = 0. We are interested 
in obtaining information about the Jacobian of the vector field / evaluated at 
(x,p), or at least about the signs of the derivatives dfi/dxj{x,p). For example, 
if dfi/dxj > 0, this means that Xj has a positive (catalytic) effect upon the rate 
of formation of Xi. The critical assumption, indeed the main point of [8,9], is 
that, while we do not know the form of /, we do know that certain parameters 
Pj do not directly affect certain variables Xi. This amounts to a priori biolog- 
ical knowledge of specificity of enzymes and similar data. This knowledge will 
be summarized by the binary matrix = {c^j) G {0, where “c? = 0” 

means that pj does not appear in the equation for ij, that is, dfi/dpj = 0. 

The experimental protocol allows us to make small perturbations in which 
we change one of the parameters, say the fcth one, while leaving the remaining 
ones constant. (A generalization would allow for the simultaneous perturbation 
of more than one parameter.) For the perturbed vector p « p, we measure the 
resulting steady state vector x = ^(p). (Mathematically, we suppose that for 
each vector of parameters p in a neighborhood of p there is a unique steady 
state ^(p) of the system, where ^ is a differentiable function. In practice, each 
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such perturbation experiment involves letting the system relax to steady state, 
and the use of some biological reporting mechanism, such as microarrays, in 
order to measure the expression profile of the variables Xi.) For each of the 
possible m experiments, in which a given pj is perturbed, we may estimate the 
n “sensitivities” 

Q^i X 

bij = — {£,i{p + PjEj) - ^i{p)) , 

apj Pj Pj 

(where Cj G K™ is the jth canonical basis vector). We let B denote the matrix 
consisting of the 6y’s. (See [8,9] for a discussion of the fact that division by 
Pj —pj, which is undesirable numerically, is not in fact necessary.) Finally, we let 
A be the Jacobian matrix df jdx and let C be the negative of the Jacobian matrix 
df tdp. From f{^{p),p) = 0, taking derivatives with respect to p, and using the 
chain rule, we get that A = BC. This brings us to the problem stated in this 
paper. (The general position assumption is reasonable, since we are dealing with 
experimental data.) 



2.2 Combinatorial Formulation of Questions (Ql) and (Q2) 

Problem Name: CPfc (the fc-Covering problem that captures Ques- 
tion (Ql) and (Q2))^ 

Instance < m,n,k >: U = [l,m] and sets Ti,T2,...,T„ C U with 
U”=iT, = U. 

Valid Solutions: A subset U' C U such that \U' H T^j > n — fc for each 
i G [1, nj. 

Objective: Minimize \U'\. 

2.3 Equivalence of CPfc and SC„_fc 

We can establish a 1-1 correspondence between an instance <m,n,k> of CPfc 
and an instance <n,m,n — k> of SC„_fc by defining Si = { j \ i & Tj} for each 
t G [1, to]. It is easy to verify that U' is a solution to the instance of CPfc if and 
only if the collection of sets Su for each u G U' is a solution to the instance of 
SC„_fc. 

3 Approximation Algorithms for SC;. 

An £-approximate solution (or simply an e-approximation) of a minimization 
problem is defined to be a solution with an objective value no larger than e times 
the value of the optimum. It is not difficult to see that SCfc is NP-complete even 
when k = n — c for some constant c > 0. 



^ CP„_i is known as the hitting set problem [5, p. 222]. 
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3.1 Analysis of Greedy Heuristic for SCfc for Large k 

Johnson [6] provides an example in which the greedy heuristic for some instance 
of SC over n elements has an approximation ratio of at least log2 n. This ap- 
proach can be generalized to show the following result. 

Lemma 1. For any fixed c > 0, the greedy heuristic (as described in Fact 1(b ) ) 
has an approximation ratio of at least — o(l)) ^ sn (^2 ) ^082^^ = l7(logn) for 
some instance <n,m,n — c> o/SC„_c- 



3.2 Randomized Approximation Algorithm for SCfc 

As stated before, an instance <n,m,k> of SCj, can be (1 -I- In a)-approximated 
in 0{mnk) time for any k where a = maxsg5{|S'|}. In this section, we provide a 
randomized algorithm with an expected performance ratio better than (l-|-lna) 
for larger k. Let S = {S'!, S 2 , ■ • ■ , Sm}- 

Our algorithm presented below as well as our subsequent discussions and 
proofs are formulated with the help of the following vector notations: 

— All our vectors have m coordinates with the i^'^ coordinate indexed with the 
z*'' set Si of 5. 

— if 14 c 5, then v G {0, 1}™ is the characteristic vector of V, i.e., vsi = 
( 1 a Si €V 

[OHS.^V 

— 1 is the vector of all I’s, i.e. 1 = s; 

— S'* = {Ag5: i G A} denotes the sets in S that contains a specific element 
i. 



Consider the standard integer programming (IP) formulation of an instance < 
n,m,k> of SCfc [14]: 



minimize \x subject to 



s'^x > k for each i G U 
XA G {0, 1} for each A G 5 



A linear programming (LP) relaxation of the above formulation is obtained by 
replacing each constraint a;AG{ 0 ,l}by 0<a;y4<l. The following randomized 
approximation algorithm for SC^ can then be designed: 



1. Select an appropriate positive constant P > 1 m the following manner: 

f Ina if fc = 1 

/3 = f ln(a/(fc — 1)) if a/{k — 1) > and A: > 1 
2 otherwise 

2. Find a solution x to the LP relaxation via any polynomial-time algorithm for 
solving linear programs (e.g. [7]). 

3. (deterministic rounding) Form a family of sets = {A G S : Pxa > !}• 

4. (randomized rounding) Form a family of sets C 5 — by independent 

random choices such that Pr[A G C^j = Pxa. 

5. (greedy selection) Form a family of sets as: 

while s*(c° -I- -I- c^) < fc for some i G U, insert to any A G S’’ — (uf=oC*). 

6. Return C = UC^ UC^ as our solution. 
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Let r(a, k) denote the performance ratio of the above algorithm. 

Theorem 2.^ E[r(a, A:)] is at most 

1 + In a, if fc = 1 

(l + ln(a/(fc — 1)), if a/{k — 1) > e^ and fc > 1 

min{ 2 + 2 • 2 + (e“^ + e“®/®) • | } 

« min{ 2 + 2 • 2 + 0.46 • | } if a/(k — 1) < e^ and fc > 1 

Let OPT denote the minimum number of sets used by an optimal solution. 
Obviously, OPT> la; and OPT> A proof of Theorem 2 follows by showing 
the following upper bounds on E[r(a, k)] and taking the best of these bounds 
for each value of a/k: 

1 + Ina, if fc = 1 

(l + ln(a/(A: — 1)), if a/ {k — 1) > e^ and A: > 1 

2 + 2 • if a/(A: — 1) < e^ and A: > 1 

2 + (e-2 + e-9/s) • f , if a/{k - 1) < e^ and A: > 1 



3.2.1 Proof of E[r(a, fc)] < 1 + In a if fc = 1, 

E[r(a, fc)] < (l + ln(a/ (fe — 1)) if a/(fc — 1) > e^ and fc > 1, 

and E[r(a, fc)] < 2 + 2 • if a/{k — 1) < e^ and fc > 1 



For our analysis, we first define two following two vector notations: 



,0 if PXA >1 1 ^ 

^ 0 otherwise 



0 if /3xa > 1 
XA otherwise 



Note that = [a:^] < /3x^. Thus \x^ < lc° < j3\x^ . Define bonus = (i\x^ — 
lc°. It is easy to see that E[l(c° + c^)] = filx — bonus. 

The contribution of set A to bonus is f3x^ — c^. This contribution to bonus 
can be distributed equally to the elements if A. Since \A\ < a, an element 
i G [l,n] receives a total of at least b^/a of bonus, where V = s*(/3a;° — c°) The 
random process that forms set has the following goal from the point of view 
of element i: pick at least (/* sets that contain i, where = k — s'^c^ These sets 
are obtained as successes in Poisson trials whose probabilities of success add 
to at least p* = /3(k — s’’x^). Let y* be random function denoting the number 
that element i contributes to the size of C^; thus, if in the random trials in 
Step 4 we found h sets from S'* then y* = max{0,A: — h}. Thus, E[r(a, A:)] = 
E[l(c° + + c^)] < /31a; + ~ 9* = — x^). We can 

parameterize the random process that forms the set from the point of view 
of element i as follows: 

The case of A: = 1 was known before and included for the sake of completeness only. 
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— is the goal for the number of sets to be picked; 

— p* = P{k — = /3g* + {(3 — is the sum of probabilities with which 

sets are picked; 

— b'^/a is the bonus of i, where 6* = s*(/3a:° — c°) > {(3 — l){k — g'' — <?*); 

— g* > 0, g* > 0 and g* + g* < k] 

— g* measures how much the goal is missed; 

— to bound E[r(a, k)] we need to bound E[g* — ^]. 

3. 2. 1.1 g-Shortage Functions 

In this section we prove some inequalities needed to estimate E[g* — tightly. 
Assume that we have a random function X that is a sum of N independent 0-1 
random variables Xi. Let E[A] = = 1] = M &nd g < /i be a positive 

integer. We define g-shortage function as = max{g — A, 0}. Our goal is to 
estimate E[Yg'^]. 

Lemma 2. E[F/] < XXo 

From now on we will assume the worst-case distribution of so we will 
assume that the above inequality in Lemma 2 is actually an equality (as it 
becomes so in the limit), i.e., we assume E[Y)(^] = X?=o ^ fixed 

f3, we will need to estimate the growth of E[Y)®^] as a function of g. Let Pg{P) = 

ea/^-E[Y30]. 

Lemma 3. pg{l) = XCo 

Lemma 4. For (3 > 1, is a decreasing function of j3. 

Lemma 5. If g > I and (3 > I then < e"^ 



Lemma 6. 



~e3xFV 



< e-9(i-i//3) 



3. 2. 1.2 Putting All the Pieces Together 



In this section we put all the pieces together from the previous two subsections 
to prove our claim on E[r(a, /c)]. We assume that (3 > 2 ii k > 1. Because we 
perform analysis from the point of view of a fixed element i, we will skip i as a 
superscript as appropriate. As we observed in Section 3.2.1, we need to estimate 
E[g - A] and b > {P — l){k — g — q). We will also use the notations p and q as 
defined there. 

Obviously if g = 0 then g = 0. We omit the case of A: = 1 and assume that 
A: > 1 for the rest of this section. We first consider the “base” case of g = 1 and 
g = 0. Since g = 0, c° = x^. Thus, b = s*(/3c° — c°) = (/?— l)s*c° = (/3— 1)(A:— 1). 
Next, we compute E[g]. Since p = Pg = P, E[g] = E[F/^] = e“^. 
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We postulate that 



a a 



.-/3 



< 



k-1 



(3—1 a 

^ e^(/3 - 1) > “ 



k-1 



= (3 + ln(/3 — 1) > In 



k-1 



(3) 



It is easy to see that, for the base case, E [l(c° + + c^)] < (31x < ln(a/(fc — 

l))OPT. 

Now we consider the “non-base” case when either g > 1 or q > 0. Compared 
to the base case, in a non-base case we have bonus ^ decreased by at least 
(/? - 1)(5 + q - l)/a. Also, E[y] = E[E/] = 



Lemma 7. 



< e-(9+«-i)/5. 



E[Yf 



Summarizing, when bonus is decreased by at most (P — l)(g + q — 1) / a = 
{P—l)t/a, we decrease the estimate of E[y] by multiplying it with at least 
As a function oi t = g + q — 1 we have 

E[y] - b/a < - ^^{k -l-t)= L~t/5 _ ^ ^ 

a a \ k — 1 J 

This is a convex function of t, so its maximal value must occur at one of 
the ends of its range. When t = 0 we have 0, and when t = A: — 1 we have 
(/3-i)(fc-i) g-(fc-i)/5^ result, our expected performance ratio for fc > 1 is 

given by 

E[r(a, k)] < Plx + '£i=i E[/ - 

< /30PT -h ^e" 

< /3(l-he-('=-i)/5)OPT 



< 



(1 -I- ln(a/(fc - 1)) OPT if a/{k - 1) > 



2- (l-f OPT 



if a/(fc — 1) < 
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Abstract. Given an undirected n-node graph and a set C of m cuts, the mini- 
mum crossing spanning tree is a spanning tree which minimizes the maximum 
crossing of any cut in C, where the crossing of a cut is the number of edges in the 
intersection of this cut and the tree. This problem finds applications in fields as 
diverse as Computational Biology and IP Routing Table Minimization. 

We show that a greedy algorithm gives an 0(r log n) approximation for the prob- 
lem where any edge occurs in at most r cuts. We then demonstrate that the prob- 
lem remains NP-hard even when G is complete. For the latter case, we design 
a randomized algorithm that gives a tree T with crossing 0((logm -|- logn) • 
(OPT -f log n)) w.h.p., where OPT is the minimum crossing of any tree. 

Our greedy analysis extends the traditional one used for set cover. The random- 
ized algorithm rounds a LP relaxation of a corresponding subproblem in stages. 



1 Introduction 

Given a graph G = (V,E) with n nodes and a family of cuts C = {C\, , Cm], the 
minimum crossing tree is a spanning tree T, which minimizes the maximum crossing of 
any cut, where the crossing of a cut Ci is defined as \E{T)GCi\. If the family of cuts is C 
= { (u, U \ u) : V G V}, then the MCST problem reduces to finding the minimum degree 
spanning tree problem which has been widely studied [8]. Hence, NP-completeness of 
the minimum degree spanning tree problem [7] shows that MCST problem is NP-hard. 

In this paper, we show approximation guarantees for the greedy algorithm for the 
MCST problem. 

Theorem 1. Given a graph G = (V, E) and a family of m cuts C={C'i, . . . , Gm}, a 
greedy algorithm for MCST problem gives a spanning tree T which crosses any cut in 
C at most 0{r ■ log n) times the maximum crossing of an optimal tree. 

Although the minimum degree spanning tree problem is trivial on complete graphs, 
surprisingly, the MCST problem remains difficult even for this special case. We show 
that the decision version of even this version of the MCST problem is NP-complete. 

Theorem 2. Given a complete graph G, set of cuts C and a positive integer k, the 
problem of determining whether there exists a spanning tree of G which crosses any cut 
in C at most k times, is NP-complete. 

* Supported in part by NSF grant CCR-0105548 and ITR grant CCR-0122581 (The ALADDIN 
project). 

K. Jansen et al. (Eds.): APPROX and RANDOM 2004, LNCS 3122, pp. 51-60, 2004. 

(c) Springer- Verlag Berlin Heidelberg 2004 
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A proof of the above theorem appears in the Appendix. The particular case of com- 
plete graphs finds application in helds as varied as IP routing and computational bi- 
ology. We give improved algorithm for the MCST problem on complete graph which 
gives better performance guarantees. 

Theorem 3. There is a randomized LP rounding based algorithm, which given a com- 
plete graph G and a family of cuts C={Ci, . . . , C^}, outputs a spanning tree T such 
that crossing for any cut Ci G C is 0((log m -f log n) ■ {OPT -f log n)), where OPT is 
the maximum crossing of an optimal tree. 

1.1 Motivation: Chimerism in Physical Mapping 

The MCST problem finds important applications in computational biology. The physi- 
cal mapping problem of the human genome project is to reconstruct the relative position 
of fragments of DNA along the genome from information on their pairwise overlap. One 
has a collection of clones and a set of short genomic inserts (called probes). A probe 
defines a single location where a given subset of clones coincide. For each probe/clone 
pair, it can be determined whether the clone contains the probe as a subsequence using 
biological techniques. The problem is to construct the order in which the probes would 
occur along the original chromosome that is consistent with the given the probe/clone 
incidence matrix. This can be done efficiently if there is no chimerism. Chimerism is the 
result of concatenating two or more clone from different parts of the genome, produc- 
ing a chimeric clone -one that is no longer a simple substring of the chromosome. More 
formally, the problem is as follows: Given a probe-clone incidence matrix A, with rows 
indexed by probes and columns by clones, and the entry atj is 1 iff probe i occurs in 
clone j. If there is no chimerism, then the problem is reduced to finding a permutation 
of rows so that ones in each column are consecutive (called as 1-ClP) and this can be 
solved efficiently in polynomial time [1]. However, in the presence of chimerism, the 
problem is more difficult. Then, we need to find a permutation tt of rows, such that 
each column has at most k blocks of consecutive ones (called as k-consecutive ones 
property or k-ClP), if the chimeric clones are a concatenation of at most k clones. The 
decision version of this problem i.e ’’Does a given 0-1 matrix have the k-consecutive 
ones property?” has been proven to be NP-complete for A: > 2 in [5]. 

1.2 k-ClP and Vector TSPs 

A classical way to solve the k-ClP problem is to reduce it to a particular multidimen- 
sional TSP problem called the Vector TSP (vTSP). This problem is defined on a com- 
plete graph G = {V,E), where each edge e G E is assigned an m-dimensional cost 
c : E ^ {0, 1}™. The cost of a tour T in G is the m-dimensional vector c(T) = 
J2eGE{T) objective is to minimize |jc(T)||oo. 

The reduction from fc-ClP to vTSP is straightforward. Each row of A becomes a 
node in G and the cost assigned to edge e = (i,j) is set to the XOR- vector between the 
two rows ai and aj. Now, let tt be the permutation induced by a solution T of vTSP, and 
let b{A'^) be the maximum number of blocks of consecutive ones in A^. Then, we have 
that b{A^) = Solving the vTSP problem is NP-hard by this reduction from the 



On the Crossing Spanning Tree Problem 



53 



2-ClP problem. However, since the Hamming distance obeys the triangle inequality, it 
is possible to use the standard Euler Tour short-cutting technique in order to compute a 
2r-approximate solution, given an r-approximation to the related Vector MST problem 
(vMST). 

The vMST can be formulated as the minimum crossing spanning tree problem on a 
complete graph G. Any column j of A can be seen as a cut Cj = {Vj,V\Vj) dehned on 
Gby setting = {vi € V\aij = 0}. The cost of edge e = (z,_;/)is as before the XOR- 
vector between at and aj i.e. c(e) is a 0-1 vector, where the entry corresponding to 
a cut Cl is 1 iff the edge (i,j) crosses G/. Here, the terminology that an edge e crosses 
a cut G is used interchangeably with e G C. For any tree T, let c(T) = J2eGE{T) 

The entry of the vector c(T) is exactly the number of edges of T crossing the cut 
Ci. Thus, the minimum crossing spanning tree minimizes ||c(r)||oo- 

1.3 Motivation: IP Routing 

Another useful application of the MCST problem can be found in [2] where it is shown 
that the an efficient solution for the min-fc-ClP can be used to obtain an good approx- 
imation for the Interval Routing problem: given a set of IP routing tables sharing the 
same host space, the problem is to reassign the IP addresses to the hosts in order to 
minimize the maximum size of any IP routing table. 

This IP routing table minimization problem, MIN-IP for short, can be formalized as 
follows. We are given a set i? = {ri, . . . , r„} of n routers and a set iT = {iii, . . . , hm] 
of m destination hosts. Each router Vj G R has a degree 5j, that is 5j out-edges, and a 
routing table specifying which of the out-edges to take for every host. The problem is 
to choose the IP addresses of the m hosts and construct the n IP routing tables so as to 
minimize the maximum size of a table, that is the maximum number of used entries in 
a table. 

In [2] it is shown that, given any r-approximation algorithm for the problem of 
determining a row permutation that minimizes the maximum number of blocks (of ones) 
in a boolean matrix A, an efficient 2r log m-approximation algorithm exists for MIN-IP, 
which exploits a matrix representation of the instances of the problem. 

Similar applications can be found also in designing interval routing schemes as 
proposed in [3,4]. 



1.4 Related Work 

As observed earlier, the minimum degree spanning tree problem is a special case of the 
MCST problem. The best result for the minimum degree spanning tree problem are due 
to Purer and Raghavachari [8]. They construct a spanning tree with maximum degree 
at most A* + 1 where A* is the maximum degree of the optimal tree. The vMST 
problem has been considered by Greenberg and Istrail [6]. They give solution of cost 
0(s(A) • OPT + logn). Here s(A) = max\<i<n Sj=i Note that r in Theorem 
1 is different from s(A) in [6]: r is the maximum number of cuts a given edge e can 
cross, where the cuts are defined by columns of A; s(A) is the sparsity of A i.e. the 
maximum number of 1 ’s in any row in A. Observe that r < 2 • s(A), but s( A) can be as 
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bad as m. Hence, our algorithm gives comparable or better performance guarantee than 
the algorithm in [6]. 

The paper is organized as follows. In Section 2, we describe a greedy algorithm for 
the MCST problem and prove Theorem 1. In Section 3, we give a randomized algorithm 
for the special case and prove the guarantees of Theorem 3. In the Appendix, we show 
that the MCST problem is NP-hard even for complete graphs. 

2 Greedy Algorithm for the General Case 

In this section, we show that the greedy algorithm gives an 0{r ■ logn) approxima- 
tion for the MCST problem where r is defined as maxegc \{C G C: e € C}\. Given 
any subgraph H, the maximum number of times H crosses any cut in C is denoted by 
Cross{H,C). 



Greedy Algorithm: 

4 > 

while F is not a tree 
do 

Let e' be an edge which minimizes Cross{F U e, C) 
over all edges e G G which join two components of F. 

F^ FUe' 
od 



First, we give a lower bound for the MCST problem. 

Lemma 1. Given any S G C, let k be the number of components formed after removing 
the edges from G of all cuts in S. Then 

k-l 

Proof. Any spanning tree of G must choose at least k— 1 edges to join the k components 
formed after removing the edges of cuts in S. Each of these k — 1 edges crosses at least 
one of the cuts in S. Hence, the average crossing of such a cut in S is at least 

The Proof of Theorem 1. Let the solution returned by the greedy algorithm be Tg and 
let I = Cross{Tg,C). We can divide the running of the greedy algorithm in I phases. 
The phase of the algorithm is the period when Gross{F, C) = i. Let ki denote the 
number of components in F when the phase ends. Let Mi be the cuts which are 
crossed by i edges at the end of phase and rrii = \Mi\. 

Consider the running of the algorithm in the phase. The crossing number of 
at least rrii cuts increases by 1 in the phase. Each edge can increase the crossing 
number of at most r cuts. Hence, in the phase we must include at least [— ] edges 
in F. Every edge, when included in F, reduces the number of components in F by 
exactly one. Therefore, we have the following inequality 



k^ < ki-i 

r 



( 1 ) 
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When the phase ends, every edge joining two components of F must cross at least 
one of the cuts in Mi, else the greedy algorithm would choose such an edge in the 
phase. Applying Lemma 1, we get the for each i, 

opt > — (2) 

ntii 

Using (1) and (2), we have that for each i, 

h-i -ki> — 7 (3) 

r * opt 

Using ki>2 for each i < / — 1 and ki-i > ki, we have for each i. 



h- 1 — 

^ 1 f' — 2r^opt 

>h-i + 



>ki{l 



2r^opt ^ 

1 V 

2r^opt ' 



As, ko = n and ki = 1, we get that 



> ( 1 + 

^logn>/log(l+ 2 ^) 



Using, log(l + x) > X — ^ and r * opt > 1 we get 



I < 4r log n * opt 



Hence, the greedy algorithm is a 0{r log n) approximation. 



□ 



3 A Randomized Algorithm for the Case of Complete Graphs 

In this section, we describe a randomized algorithm for MUST for complete graphs and 
prove that it gives a tree with maximum crossing 0((log m + log n) ■ (OPT + log n)) 
with high probability, where n is the number of vertices in G and m is the number of 
cuts in C. 

The idea is the following : Start with each vertex as a different component and merge 
components in phases until a connected subgraph is obtained. In a phase, each compo- 
nent is represented by an arbitrarily chosen vertex of the component. We carefully se- 
lect some edges between the representative vertices by solving a multicommodity flow 
problem in each phase, so that the cuts in C are not crossed “too much”. We ensure that 
at least one edge is chosen out of each representative in every phase. Hence, the num- 
ber of components reduces by at least a factor of two and thus a connected subgraph is 
obtained in at most log 2 n phases. 

In phase p, we solve the following multicommodity flow problem on a graph G' 
constructed from a complete graph Gp (on the representative vertices in this phase) as 
follows. Let U(Gp) = {vi,V 2 , ■ ■ ■ ,Vn^}. 
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- For each undirected edge e = {u,v), add two directed edges e/ = (u,v) and 
€r = {v, u) in G' , 

- For each vertex Vi G V{Gp) introduce a new vertex in ^ (G') and 

- Vfj G V{Gp),j ^ i, add the directed edge (t;^, in G'. 

Now, the flow problem on G' is the following. Each vertex Vi G V (G') is required 
to send a unit flow of commodity i to . Let fi, f 2 , ■ ■ ■ , fn be the flows associated 
with each of the n commodities. Let fi{v) denote the net flow of commodity into 
the vertex v. The following integer program accomplishes our goal. 

min z 

s.t. 2 — J2eeE(G)nC yC G C 

= J2i=l,...,np fi(^) 

Vi 1 , . . . , Tip 

fi(v) yvGV(G') 

Mv) =0 Vt;€ y(G')\{t;,,s„J 

fi{vt) = 1 

fi{Svi) = -1 

Me) G {0, 1} Ve G E{G') 

We now describe the algorithm for the MCST problem. We will construct a con- 
nected subgraph H with a low maximum crossing. 

1. Initialize V{H) ^ V{G), E{H) ^ 4>, Go ^ G, Rq ^ V{G), p ^ 0. 

2. While H is not connected 

(a) Construct G' from Gp. Solve the LP-relaxation of the corresponding integer 
program for phase p and obtain an integral solution X by randomized rounding 
of the optimum LP solution [10]. 

(b) Let E' = {e G Gp : Xe > 0}. E{H) ^ E{H) U E' . 

(c) p <— p + 1. Let Rp be the set of representative vertices (chosen arbitrarily one 
for each connected component of H), Gp is the complete graph on the vertices 
of Rp. 

Let T* be a optimal tree for the MCST problem and let OPT be the maximum cross- 
ing of any cut in T* . 



Proposition 1. Let z* be the optimum to the LP-relaxation in phase p. Then z* < 
20PT. 

Proof. We can construct a feasible solution of the LP from the optimum tree T* of 
value at most 20PT. Let Ri = {v \, . . . , f^p} be the set of representatives in phase i. 
Lrom the Tree Pairing Lemma in [9], there exists a matching M between vertices of 
Ri such that the paths in T* between the matched pairs of vertices are edge disjoint. 
We can use this matching to construct a feasible solution to the LP. Send a unit flow 
of commodity i on the directed path Pvi.vj U {vj , ) and of commodity j on the path 

Pvj.vi U {vi, Syj), where P(u, v) is the unique path in tree T* between matched pairs u 
and V. The above flow is a feasible flow as it satisfies all the flow constraints of the LP. 
Every edge of T* carries at most two units of flow. Flence, the objective value 2 ; for this 
feasible flow, is at most 20PT. Therefore, z* < 20PT. 
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Proposition 2. If an edge e = (m, v) crosses a cut C, then any other path between u 
and V also crosses the cut C at least once. 

Proof. If we remove all the edges in C from G, then u and v would he disconnected. 
Thus, every path from utov contains an edge of C. 

We will use Proposition 2, to obtain and work with a special kind of optimum so- 
lution such that each flow path uses only two edges. Consider the flow decomposition 
for commodity i in the optimum solution of the LP-relaxation and consider a flow path 
P =< . ■ ■ , Vi^, , Sy^ >. We can replace P by the path P' =< Uj, Vi^, , > 

without increasing the maximum crossing. From Observation 2, we know that any cut 
that the edge crosses will be crossed at least once by the path P. Therefore, 

P' only reduces the number of crossings for the cuts in C and so we can replace P by 
P' . Thus, we can obtain a fractional optimum solution S* such that each flow path uses 
only two edges. This step greatly simplifies the subsequent analysis of the randomized 
rounding since every cut crosses every flow path at most once after this preprocessing. 



3.1 Rounding S* to an Integral Solution 

Let us describe the rounding of the fractional multicommodity flow obtained by solving 
the LP relaxation corresponding to phase p. The flow corresponding to each commodity 
is rounded independently of others. For each commodity i = 1, . . . , Up, choose an edge 
e = {vi, Vj) with probability fi{vi, vj). The corresponding flow is routed through the 
path < Vi , Vj , Sy^ > and the edge (vi , Vj ) is included in the subgraph H. This is repeated 
for every commodity independently. 

In phase p, let the fractional optimum flow be f* and the optimum LP solution be 
z*. Let z{C) denote the number of edges crossing a cut C G C. Consider Yj, a 0-1 
variable associated with the commodity, where 

y r 1 if the integral flow crosses C 
^ ^0 otherwise 



Therefore, 



Pr{Y, = 1) = ^ /,(e) 

eeE{Gp)nC 

Tip 

z{C) = ^Y, 

i=i 

E[z{C)\ = SeGB(Gi)nC /i(®) 
= SeGB(Gi)nC Si=l /j (®) 

= Z)eG£;(Gi)nC 

<5 < 2 • OPT 



z{C) is the sum of independent Bernoulli trials. Thus, we can use Chernoff bounds 
to bound the tail probability 



2E[z{C)] 



Pr(\z{C) - E[z{C)]\ > k(3) < exp{ 
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Let (3 = E[z{C)\ + log n and k = log„ m + 2. Therefore, 



Pr{\z{C) - E[z{C)] \ > kP) < exp{ 

< exp{ 



(E[z(C)\+logn) 

2 

(2 log^ m+4) log n • 
2 



1 

ran- 



Since E[z{C)\ < 20PT, we have that Pr{z{C) > {2{k + l)OPT + fclogn)) < 
^ or Pr{z{C) > 2(log„ to + 3) • OPT + (log„ to + 2) • logn) < for any 



cut C G C in any phase p. We say that a “bad” event occurs in a phase p if some 
cut C G C has a high crossing in that phase. Thus, from the union bound we have 
Pr(bad event occurs in phase p) < The algorithm has at most log 2 n phases. Thus, 



Thus, we have shown that in every phase the crossing of every cut is 0((log„ to + 
3)OPT + (log„ TO + 2) • logn) with high probability. Hence, we obtain a solution of 
maximum crossing 0((log2 to + log 2 n) ■ (OPT + log 2 n)) with probability at least 



Remark: For m > n, setting k = yj {lognm + 2) gives a slightly better solution 
with maximum crossing 0{^/JogrnJo^{0PT + log n)). 

4 Future Work 

We believe that better performance ratios can be obtained particularly for the MOST 
problem on complete graphs. Furthermore, more sophisticated methods than a simple 
greedy approach should be able to remove the factor of r in the general case. 
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Appendix: MCST for Complete Graphs Is NP-Hard 

In this section, we consider the MCST problem for complete graphs. We show that the 
problem is NP-hard even for this special case. In fact, we show that the decision version 
of the problem is NP-complete. 

Clearly, the decision problem is in NR We reduce the 2-consecutive ones problem, 
2-ClP, to MCST. Given a n x m matrix A, 2-ClP is the problem of determining whether 
there exists a permutation of rows such that in each column all ones occur in at most 2 
consecutive blocks. This problem has been shown to be NP-complete in [6]. 

Given any arbitrary nxm matrix A, make a complete graph G over n+1 vertices, with 
one vertex corresponding to each row and a new dummy vertex s. For each column in A, 
include a cut in C naturally defined by the column: vertices with rows with 1 form one 
side of the cut. The dummy vertex s is always on the 0-side of each cut. Also include in 
C singleton cuts, = ({n}, V \ {t;}) for every vertex in G. For each pair of vertices 
u and V, include in C the cut Guv = ({u, t;}, 17 \ {u, t>}). Finally, let k = A. 

We first show that if there exists a permutation of rows, tt, such that it has 2-Cl 
property, then there exists a spanning tree which crosses each cut in C at most four 
times. Consider the Hamiltonian path H which starts at s and then traverses the vertices 
in the order corresponding to permutation tt. Each cut corresponding to a column is 
crossed by the Hamiltonian path H exactly when the row permutation tt switches from 
a row with 0 with a row with 1 or vice versa. As all the ones are in 2 consecutive blocks, 
each cut can be crossed at most four times. Introducing the dummy node corresponds 
to introducing a row with all zeros as the first row which clearly does not change 2-Cl 
property. Also, a Hamiltonian path crosses each singleton cut at most two times and 
cut Guv at most two times for any u,v G V. Hence, there exists a spanning tree which 
crosses every cut in C at most four times. 

Now, for the other direction we show that if there exists a spanning tree T which 
crosses every cut in C at most 4 times then there exists that a permutation tt which has 
the 2-ClP property. We claim that any such tree must be a Hamiltonian path. As each 
singleton vertex is a cut in C, hence degree of each vertex is at most four. Suppose there 
exists a vertex u with degree four. For n > 5, there exists a vertex v which is not a 
neighbor of u. But, then the cut Gw is crossed at least five times. Hence, all vertices 
have degree at most three. Suppose, for the sake of contradiction there exists a vertex 
u such that degriu) = 3. Consider any vertex v which is not a neighbor of u. As T 
crosses Cw at most four times, so degriv) = 1. This implies that the total sum of 
degrees of nodes in T is at most 3 * 4 + (n — 3). Hence, 2n — 2< n + 9 or equivalently, 
n < 11 which is a contradiction assuming larger n. Hence, every vertex must have 
degree at most two in T showing that T is a Hamiltonian path. 



60 



Vittorio Bilo et al. 



Let the hamiltonian path be {vi, . . . ,Vk, s, Vk+i,Vn)- Consider the following per- 
mutation of rows {rk+i, . . . ,rn,ri, . . . ,rk) where Vi corresponds to row in the trans- 
formation. We claim that in each column, there cannot be more than two blocks of ones. 
Suppose for the sake of contradiction, there exists a column Ci which has three blocks 
of ones. Thus, the cut corresponding to the Hamiltonian cycle formed by joining 
and vi must cross the cut corresponding to column Ci at least hve times. But any cycle 
crosses any cut even number of times. Hence, it must cross the cut at least six times, but 
then the hamiltonian path must cross the cut at least five times, a contradiction. Hence, 
there exists a permutation which satisfies the 2-Cl property. This reduction shows that 
decision version of MCST problem for complete graphs is NP-complete. □ 
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Abstract. We present a polynomial time 3/4-approximation algorithm 
for the maximum asymmetric TSP with weights zero and one. 

As applications, we get a 5/4-approximation algorithm for the (mini- 
mum) asymmetric TSP with weights one and two and a 3/4-approxi- 
mation algorithm for the Maximum Directed Path Packing Problem. 



1 Introduction 

Traveling salesperson problems with weights one and two have been studied 
for many years. They are an important special case of traveling salesperson 
problems with triangle inequality. Papadimitriou and Yannakakis [8] showed that 
the undirected minimization problem is MaxSNP-complete. On the other hand, 
they presented a 7/6-approximation algorithm with polynomial running time. 
Vishwanathan [9] considered the corresponding asymmetric problem ATSP (1,2) 
and gave a 17/ 12- approximation algorithm. 

Let MaxATSP(0, 1) be the following problem: Given a directed complete loop- 
less graph with edge weights zero and one, compute a TSP tour of maximum 
weight. MaxATSP(0, 1) is a generalization of ATSP(1,2) in the following sense: 
Vishwanathan [9] showed that any (1 — o;)-approximation algorithm for the for- 
mer problem transforms into an (1 -P a)-algorithm for the latter when replacing 
weight two with weight zero. (The other direction is not known to be true.) 

By computing a matching of maximum weight and patching the edges to- 
gether arbitrarily, one easily obtains a polynomial time 1/ 2-approximation al- 
gorithm for MaxATSP(0, 1). (Note that each edge has weight at least zero, thus 
we cannot loose any weight during the patching process.) Vishwanathan [9] was 
the first to improve on this by designing a 7/ 12-approximation algorithm with 
polynomial running time. In 1994, Kosaraju, Park, and Stein [6] gave a 48/63- 
approximation algorithm with polynomial time that also worked for maximum 
ATSP with arbitrary nonnegative weights. In their work, they also formulated 
the so-called path coloring lemma, which will be crucial for our algorithm. 
Blaser and Siebert [3] obtained a 4/3-approximation algorithm with running 
time for ATSP(1,2). This algorithm can also be modified to give a 2/3- 

approximation algorithm for MaxATSP(0, 1) with the same running time [4]. Fi- 
nally, Kaplan et al. [5] generalize this result by designing a 2/ 3- approximation 
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algorithm that works for maximum ATSP with arbitrary nonnegative weights 
but has a worse running time. 

Closely related to MaxATSP(0, 1) is the Directed Path Packing Problem DPP. 
Here we are given a directed graph G = {V,E). The aim is to find a subset P 
of node-disjoint paths of G such that the number of edges in P is maximized. 
By giving edges in G weight one and “non-edges” weight zero, any path packing 
transforms into a TSP tour by patching the paths arbitrarily together. On the 
other hand, any TSP tour yields a path packing by discarding all edges of weight 
zero. The only exception is the case where an optimum TSP tour has weight n. 
Here one weight one edge has to be discarded. 

Our main result is a 3/4-approximation algorithm for MaxATSP(0, 1) with 
polynomial running time. As corollaries, we get a 5/4-approximation algorithm 
for ATSP(1,2) and a 3/4- approximation algorithm for DPP. 

1.1 Notations and Conventions 

For a set of nodes V, K{V) denotes the set of all edges (u,v) with u yf n. In 
other words, (V,K{V)) is the complete loopless graph with node set V. 

For a multigraph H and an edge e of H, mnie) G N denotes the multiplicity 
of e in H, that is, the number of copies of e that are present in H. If H is clear 
from the context, we will also omit the subscript H . 

A multigraph is called 2-path- colorable if its edges can be colored with two 
colors such that each color class is a collection of node-disjoint paths. (Double 
edges are not allowed in such a collection of paths.) 

1.2 Outline of Our Algorithm 

Kosaraju, Park, and Stein [6] formulate the so-called path coloring lemma. It 
states that if each node of a multigraph H has indegree and outdegree at most 
two and total degree at most three and PI does not contain any 2-cycles (that 
is, a cycle with exactly two edges) or triple edges, then P[ is 2-path colorable. 
Kosaraju, Park, and Stein proceed with computing a cycle cover and a matching. 
(A cycle cover of a graph is a collection of node-disjoint directed cycles such that 
each node belongs to exactly one cycle.) If the matching is carefully chosen, then 
the union of the cycle cover and the matching fulfills the premises of the path 
coloring lemma and henceforth, is 2-path-colorable. (One also has to deal with 
the 2-cycles in the cycle cover separately, the interested reader is referred to the 
original work.) If one now takes the color class with the larger weight and patches 
the paths arbitrarily together, one gets a TSP tour that has at least half the 
weight of the combined weight of the cycle cover and the matching. The weight 
of an optimum cycle cover is at least the weight of an optimum TSP tour and the 
weight of an optimum matching is at least half the weight of an optimum TSP 
tour. Thus in the ideal case, this would yield an 3/4-approximation. However, 
Kosaraju, Park, and Stein have to deal with 2-cycles and have to avoid triple 
edges. Therefore, they only get a 48/63-approximation. This approach is refined 
in subsequent works [2,7]. 
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In this work, we directly compute a maximum weight multigraph that fulfills 
the premises of the path coloring lemma. This is done via an LP approach 
(Section 3). The fractional solution H* is then rounded to an integer one via 
an iterated decomposition scheme (Section 4). Finally the integer solution is 
transformed into a TSP tour (or Path Packing) via the path coloring lemma. 

2 Admissible Multigraphs 

A directed multigraph is called admissible, if 

1. the indegree and outdegree of each node is at most two, 

2. the total degree of each node is at most three, 

3. between each pair of nodes, there are at most two edges (counted with mul- 
tiplicities). 

Let uj : K{V) {0, 1} be a weight function. (This will be the weight function of 
the input of our algorithm for MaxATSP(0, 1).) Our goal is to find an admissible 
multigraph H of maximum weight where the edges are weighted according to uj, 
that is, each edge contributes weight mnie) • ca(e). 

Lewenstein and Sviridenko [7], by reduction to the path coloring lemma of 
Kosaraju, Park, and Stein [6] (see Blaser [2] for a proof), show the following 
variant of the path coloring lemma: 

Lemma 1 (Path coloring lemma). If there are no 2-cycles on a cycle in an 
admissible multigraph G, then G is 2 -path- colorable. 

Above, a 2-cycle on a cycle is a directed cycle v\, . . . ,Vk,v\ with fc > 3 such 
that (vi, Vi-i) (if i = 1, then i — 1 is fc) is also an edge for some i. 

We first compute an admissible multigraph of maximum weight. Then we 
have to remove 2-cycles on a cycle. In the case of weights zero and one, we are 
able to remove these 2-cycles without any loss of weight. 

3 LP for Maximum Weight Admissible Multigraphs 

For each edge e G K{V), there is a variable Xe- If e = (u,v), we also write Xuv 
instead of Xe- The following integral LP solves the problem of finding a maximum 
weight admissible multigraph: 

Maximize uj(e)xe subject to 

eeK{V) 



^ ^ ^UV ^ 2 


for all w G P 




(indegree) 


u^v 








^ ^ ^UV ^ 2 


for all u G P 




(outdegree) 


v^u 








^ ^ ^UV ^ ^ ^WU ^ 3 


for all u G P 




(total degree) 


v^u w^u 








^UV ^VU ^ 2 


for all u,v G V, 
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(triple edge) 


^uv € {Oj I5 2} 


for all u,v G V, 
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(multiplicity) 
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Constraints (outdegree) and (indegree) assure that the outdegree and indegree 
of each node are at most two. By (total degree), each node has total degree at 
most three. Constraint (triple edge) forbids more than two edges (counted with 
multiplicities) between each pair of nodes. The last condition ensures that each 
edge has integral multiplicity. By (triple edge), each edge can have multiplicity 
at most two. We now relax the integrality constraint: 

0 < Xuv < 2 for all u,v G V, u V (multiplicity^) 

(Note that only the lower bound of (multiplicity') is actually needed, the upper 
bound also follows from (triple edge).) Let x*^ be an optimal solution of the 
relaxed LP. Let w* be its weight. We may assume that the x*^ are rational num- 
bers. Their smallest common denominator D can be represented with poly(n) 
bits. We define the multigraph H* as follows. For all u and v with u ^ v, there 
are Xuv • D copies of the edge (u, v) in H* . Furthermore, we define the bipartite 
multigraph B* with bipartition V UV' {V' is a copy of V) as follows: If there 
are d copies of the edge (m, v) in H* , then there are d copies of the edge (m, v') in 
B*. The graph B* will be helpful in the decomposition of H*. By construction, 
H* and B* fulfill the following properties: 

(PI) The outdegree of each node v in H* is at most 2D. The degree of each 
node u G P in i?* is at most 2D. 

(P2) The indegree of each node v in H* is at most 2D. The degree of each node 
u' G V' in B* is at most 2D. 

(P3) The total degree of each node v in H* is at most 3D. The sum of the 
degrees of v and v' in B* is at most 3D. 

(P4) Between each pair of distinct nodes u and v there are at most 2D edges 
in H*. For any pair of distinct nodes u and v, the sum of the number of 
edges between u and v' and between v and u' is at most 2D. 



4 A Rounding Procedure for H* 

We first assume that D = 2^ Is a power of two. We will deal with the general 
case later. We now decompose H* into two subgraphs iJ* and such that 
both HI and fulfill the properties (P1)~(P4) of the preceding section with 
D replaced by D/2. If we now replace H* with the heavier of H/ and and 
proceed recursively, we will end up with an optimum admissible multigraph after 
log D = poly(n) such decomposition steps. 

In the following, H denotes the graph of the current iteration of the pro- 
cess outlined above and B is the corresponding bipartite graph. Our rounding 
procedure uses the fact that the edge weights are either zero or one. 

Alon [1] gives a similar procedure for edge-coloring bipartite multigraphs. 
This procedure is then used by Kaplan et al. [5] for decomposing a fractional 
solution of an LP for computing cycle covers. 
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4.1 Normalization of H 

We first ensure that H fulfills also the following property (P5) in addition to 
(P1)-(P4). 

(P5) For all u and v. If m{u,v) + m{v,u) = 2D, then m{u,v) and m{v,u) are 
both even. 

Algorithm 1 takes a multigraph H that fulfills (P1)-(P4) and transforms it 
into a multigraph H' with the same weight that fulfills (P1)-(P5). 

The algorithm basically works as follows: Let u and v be two nodes. W.l.o.g. 
m{u,v) > m{v,u). Furthermore, assume that m{u,v) + m{v,u) = 2D and both 
m{u, v) and m{v, u) are odd. (As D is even, if one of m{u, v) and m{v, u) is odd, 
then both are.) Since Z? is a power of two, m{u,v) > D. 

If u>{u,v) = 0, then we can simply remove all copies of {u,v). This does not 
change the weight of H. Furthermore, this does not affect (P1)-(P4), too. If 
uj(v, u) = 0, then we remove all copies of (v, u). The interesting case is uj(v, u) = 
oj(u,v) = 1. Here we remove one copy of (u,v) and add one copy of {v,u). This 
does not change the weight. Since m{u, v) > D, m{u, v) > D thereafter. In 
particular, the indegree of u and the outdegree of v are both still at most 2D. 
Therefore, the resulting graph H' fulfills (P1)-(P4), too. 

By construction, H' fulfills (P5). This shows the following lemma. 

Lemma 2. Given a multigraph H fulfilling (P1)-(P4) for some D = 2^ , Algo- 
rithm 1 computes a multigraph H' that has the same weight as H and fulfills 
(P1)-(P5) for D. 



4.2 Decomposition of H 

Next we describe how to decompose a graph H fulfilling (P1)-(P5) for D into 
two graphs Hi and H 2 fulfilling (P1)-(P4) for D/2. Let Hi be the heavier of Hi 
and H 2 . We thereafter normalize Hi and proceed recursively. 

In the first for loop of Algorithm 2, we run through all edges {u,v). If the 
multiplicity of ( m , v) is even in H, then we simply divide all the copies evenly 
between Hi and H 2 . If the multiplicity is odd, then we divide all but one copy 
evenly between Hi and H 2 . Thereafter, all edges in H have multiplicity zero or 
one. 

If the indegree and outdegree of a node u in H are both odd, then we add 
the edge {u,u') to B. Then we take two nodes of odd degree in a connected 
component of B. (If one such node exists, then there must exist another one.) 
We compute a path P connecting two such nodes and add the edges of P in 
an alternating way to iLi and H 2 . If there are not any nodes of odd degree, 
each connected component of B is Eulerian. We compute Eulerian tours and 
distribute the edges in the same way as in the case of a path P. 

We claim that Hi and H 2 fulfill (P1)-(P4) with parameter D/2. For (PI) 
note that we always remove the edges in pairs. If for instance an edge {u,v) is 




66 



Markus Blaser 



Algorithm 1 Normalization of H 

Input: Multigraph H fulfilling (P1)-(P4) for D = 2^ 

Output: Multigraph H' fulfilling u>{H) = and (P1)-(P5) for D 

for all unordered pairs of nodes u,v € V , u ^ v do 

Let m be the multiplicity of (ti, v) in H and m! be the multiplicity of (u, u) in H . 
W.l.o.g. m > m' . 

if m + m' < 2D or m or m' is even then 

insert {u,v) and (u, m) with multiplicities m and m! into H' , respectively, 
else 

if uj(u,v) = 0 and u>{v,u) = 1 then 
insert m' copies of (v,u) into H' . 

end if 

if Lu(u,v) = 1 and lu ( v , u ) = 0 then 
insert m copies of (u,v) into H' . 

end if 

if o)(m, v ) = oj ( v , u ) — 1 then 

insert m — 1 copies of (u, v) insert m' + 1 copies of (v, u) into H' 

end if 
end if 
end for 



removed from H and added to Hi, then also an edge (u,w) is remove from H 
and added to H 2 - In other words, the outdegree of u in Hi and i?2 is always 
the same. The only exception is the case when the original outdegree of u is 
odd. Then one more edge is added to Hi, say, than to H 2 - However, since the 
outdegree in H was odd, it was at most D — 1. Therefore, the degree in Hi is at 
most D and in H 2 , it is at most D — 1. The same argument works if the roles 
of Hi and H 2 are interchanged. In the same way, we can show that Hi and H 2 
fulfill (P2) with D/2. 

For (P3), we distinguish four cases, depending on the parity of the indegree i 
and outdegree o of a node u in iL . If both i and u are even, then the indegree and 
outdegree of u are i/2 and o/2, respectively, in both Hi and H 2 by construction. 
If both i and u are odd, then the indegree and outdegree are [z/2j and [o/2], 
respectively, in one of Hi and H 2 , and |"i/2] and [o/2j in the other of Hi and 
H 2 . This is due to the fact that we added the edge {u,u') to B. In both cases, 
the indegree and outdegree sums up to {i + o)/2 < 3H/2. Finally, we consider 
the case where either z or o is odd. We assume that z is odd, the other case is 
treated symmetrically. Then the indegree of u in Hi, say, is [z/2] and in H 2 , it 
is [z/2j. In both subgraphs, the outdegree of u is o/2. The total degree of u in 
H is however at most 3D — 1, since 3D is even. Therefore, z + o < 3D — 1 and 
[z/2] +0/2 < 3D/2. 

It remains to show that (P4) holds: Let m be the multiplicity of (zz, v) in 
H and m' be the multiplicity of (v,u) in H. If both m and m' are even, then 
half of the copies of both (zz, z;) and (z;, zz) is added to Hi and the other half are 
added to H 2 in the first for loop. Thus the number of edges between zz and v 
is (m + m')/2 < 2D /2 in both Hi and H 2 - If m is odd and m! is even or vice 
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Algorithm 2 Decomposition of H 

Input: Multigraph H fulfilling (P1)-(P5) for some D — 2^ 

Output: Multigraphs Hi and H 2 fulfilling (P1)-(P4) for D/2 
for all (ordered) pairs of nodes u and v, u ^ v do 
Let m be the multiplicity of (u,v). 

Let h = (m/2j. 

Remove 2h copies of (u, v) from H . 

Add h copies to Hi and h copies to H 2 - 
Update B accordingly. 

end for 

for all nodes u do 

if the indegree and outdegree of u in iL are both odd then 
add (u, u') to B 

end if 
end for 

while B contains nodes of odd degree do 

Choose two nodes a,h £V VJV' oi odd degree in B that lie in the same connected 
component. 

Compute a path P from a to b. 

Remove the edges of P from H and add them to Hi and H 2 in an alternating way. 
Skip all edges of the form {u,u). Update B. 

end while 

for each connected component C ot B do 
Compute a Eulerian tour of C. 

Remove the edges of C from H and add them to Hi and H 2 in an alternating way, 
again skipping all edges of the form (u, «'). 

end for 



versa, then there are [(m -I- m')/2\ between u and v in both H\ and H 2 after 
the first for loop. Then one further copy is added to either Hi and H 2 - But since 
2D is even, [{m + m')/2\ < 2D/2 and thus \{m + m')/2\ -I- 1 < 2D/2. The last 
case is the one where both m and m! are odd. Then [m/2j -|- [m'/2j copies are 
added to Hi and H 2 , respectively, in the first for loop. In the remaining steps 
of Algorithm 2, two further copies are added to Hi or H 2 - In the worst case, 
they both go to one graph, say Hi. Since H fulfills (P5), m + m' < 2D. Thus 
[m/2j + [to'/2J < D — 1. Therefore, [m/2j -|- [m'/2j -|- 2 < D = 2D/2. Thus 
Hi and H 2 also fulfill (P4) for D/2. Thus we obtain the next result. 

Lemma 3. Given a multigraph H that fulfills (P1)-(P5) for some D = 2^, 
Algorithm 2 computes two multigraphs Hi and H 2 such that both Hi and H 2 
fulfill (P1)-(P4) and for each edge e, mnie) = mHi{e) -l-m// 2 (e). 

4.3 An Algorithm for Maximum Weight Admissible Multigraphs 

Algorithm 3 repeatedly takes the multigraph H , normalizes it via Algorithm 1, 
decomposes it via Algorithm 2, and proceeds iteratively with the heavier of Hi 
and H 2 . Lemmas 2 and 3 immediately prove the following result. 
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Algorithm 3 Maximum weight admissible subgraph 

Input: Multigraph H fulfilling (P1)-(P4) for some even D = 2^ 
Output: Maximum weight admissible multigraph S 
for i = 1, . . . , (5 do 

Normalize H via Algorithm 1. 

Compute multigraphs Hi and H 2 from H via Algorithm 2. 

Let w.lo.g. Hi be the heavier of the two computed multigraphs. 
Set H = Hi. 
end for 
Set S = H. 



Lemma 4. Given a multigraph H that fulfills (P1)-(P4) for some D = 2^ and 
uj{H) = u>* , Algorithm 3 computes a maximum weight admissible multigraph S. 



4.4 Making D a Power of Two 

If D is not a power of two, then we use the following standard trick and replace 
it by = 2^ where 5 is the smallest natural number such that 2n^D < 2^ . Each 
value of the optimal solution is rounded down to the next multiple of 2~^ . 
Let Xuv be the values obtained. We have 



> X* - 2 



-s 



( 1 ) 



Let H be the multigraph that has Xuv ■ D copies of each edge {u, v). 

Since we round each value down, H fulfills (P1)-(P4). It remains to estimate 
the loss of weight. By (1), 



w{H) = ^ uj{u,v)xuv‘2^ 

{u,v)eK(V) 

>oj*2^- oj{u,v)2-^ -2^ 

{u,v)eK(V) 

\ * (~i6 2 

> oj 2 — n , 



because u){u,v) G {0,1}. If we now run Algorithm 3 on the graph H for <5 
iterations, then we end up with an admissible multigraph S of weight 

u;(S') > {uj*2^ - n^)/2^ > w* - 1/(2D). 

Therefore, Duj{S) > Du;* — 1/2. Since both quantities Du;{S) and Du* are 
integers, we have that even Du{S) > Du* . 

Therefore, we have a polynomial time solution for computing maximum 
weight admissible multigraphs. 

Theorem 1. On input H, Algorithm 3 computes a maximum weight admissible 
multigraph in polynomial time. 
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Fig. 1. On the lefthand side: A potential 2-cycle on a cycle. On the righthand side: 
The two ways how a potential 2-cycle on a cycle is treated 



5 Coloring Admissible Multigraphs 

We show that for any admissible multigraph G, there is another admissible 
multigraph G' of the same weight that is even 2-path colorable. Given G, G' can 
be found in polynomial time. Our aim is to exploit the path coloring result by 
Lewenstein and Sviridenko. To do so, we have to deal with 2-cycles on a cycle. 

A 2-cycle c on a cycle locally looks as depicted on the lefthand side of Figure 1. 
If e has weight zero, then we can simply discard e. This does not change the 
weight. The resulting graph is still admissible and c is no longer a 2-cycle on a 
cycle. If e has weight one, then we remove / and add another copy of e. This 
can only increase the weight, since the weight of / is either zero or one. The 
resulting graph is still admissible and c is no longer a 2-cycle on a cycle. (Note 
that the procedure of Lewenstein and Sviridenko can deal with double edges.) 

We now consider all (unordered) pairs of nodes and deal with them as de- 
scribed above. This shows the following result. 

Lemma 5. Given an admissible multigraph G and a weight function ui : K{V) 
—> {0, 1}, there is a 2-path- colorable admissible multigraph G' with oj(G) < 
u>{G'). Given G, G' can be found in polynomial time. 

6 Applications 

Algorithm 4 now computes a maximum path packing or a maximum TSP tour, 
respectively. We first solve the fractional LP, round the optimum solution as in 
Section 4.2, make it 2-path-colorable, and finally take the color class of larger 
weight as a path packing, in the case of DPP, or patch the paths of this class 
together to form a TSP tour, in the case of MaxATSP(0, 1). 

Theorem 2. Algorithm 4 is a polynomial time 3 / A- approximation algorithm for 
DPP and MaxATSP(0, 1). 
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Algorithm 4 DPP, MaxATSP(0, 1) 

Input: Directed Graph G = {V,K{y)) with weight function w : K(V) {0, 1} 
Output: A path packing or TSP tour, respectively 
Solve the fractional LP in Section 3. 

Round the optimum fractional solution H* to an admissible multigraph S via Algo- 
rithm 3. (Replace H* by H, if necessary.) 

Find a 2-path colorable admissible graph S' with the same weight as S. 

Color the edges of S' with two colors such that each color class is a collection of 
node-disjoint paths. 

Return the collection of larger weight. In the case of MaxATSP(0, 1), patch these path 
together arbitrarily. 



Proof. By construction, the output of the algorithm is a feasible solution and 
it is computed in polynomial time. It remains to estimate the approximation 
performance. 

If G has a path packing P with ^ edges, then there is an admissible multigraph 
of weight |t'. To see this, we decompose P into two matchings Mi and M2 by 
placing the edges of each path in P into Mi and M2 in an alternating fashion. 
Let w.lo.g. be Mi the matching with more edges. Then P U Mi is an admissible 
multigraph of weight |^. 

In the same way we see that if there is a TSP tour of weight £, there is 
an admissible multigraph of weight |^. (Strictly speaking, this is only true if 
the number of nodes is even. We can make it even by either adding a dummy 
node that is connected with weight zero edges to all other nodes. This gives a 
multiplicative loss of (1 — 1/n) in the approximation factor. Or we can guess two 
consecutive edges and contract them into one. This increases the running time 
by a quadratic factor, but does not affect the approximation performance.) 

The optimum admissible multigraph is divided into two color classes. There- 
fore, the heavier of the two classes has weight at least j£. □ 

Corollary 1. There is a £>/ 4:- approximation algorithm with polynomial running 
time for ATSP(1,2). 

Proof. Vishwanathan [9] showed that any (1 — a)-approximation algorithm for 
MaxATSP(0, 1) yields an (1 + a)-approximation for ATSP(1,2), too. □ 
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Abstract. We study a variant of the maximum coverage problem which 
we label the maximum coverage problem with group budget constraints 
(MCG). We are given a collection of sets S = {Si, S 2 , ■ ■ ■ , Sm} where 
each set Si is a subset of a given ground set X. In the maximum cov- 
erage problem the goal is to pick k sets from S to maximize the cardi- 
nality of their union. In the MCG problem S is partitioned into groups 
Gi,G 2 , ■ ■ ■ ,Gi. The goal is to pick k sets from S to maximize the car- 
dinality of their union but with the additional restriction that at most 
one set be picked from each group. We motivate the study of MGG by 
pointing out a variety of applications. We show that the greedy algorithm 
gives a 2-approximation algorithm for this problem which is tight in the 
oracle model. We also obtain a constant factor approximation algorithm 
for the cost version of the problem. We then use MGG to obtain the first 
constant factor approximation algorithms for the following problems: (i) 
multiple depot fc-traveling repairmen problem with covering constraints 
and (ii) orienteering problem with time windows when the number of 
time windows is a constant. 



1 Introduction 

In this paper we are interested in a variant of the set cover problem and its 
maximization version, the maximum coverage problem. The set cover problem 
is the following: we are given a ground set A of n elements and a set of subsets 
S = {81,82 , . . . , •S'm} such that for 1 < z < m, C X. The objective is to find a 
minimum number of sets from S such that their union is A. In the cost version 
of the set cover problem each set 8i has a cost c(S'i) and we seek a minimum 
cost collection of sets to cover all the elements of A. The maximum coverage 
problem is a close relative of the set cover problem. We are given a ground set 
A and 8 = {S'!, 82, ■ ■ ■ , 8m} of subsets of A. In addition we are given an integer 
k and the goal is to pick at most k sets from S such that the size of their union 
is maximized. In the profit version of the maximum coverage problem the items 
in A have profits and the goal is to maximize the profit of items picked. 



K. Jansen et al. (Eds.): APPROX and RANDOM 2004, LNCS 3122, pp. 72-83, 2004. 
(c) Springer- Verlag Berlin Heidelberg 2004 
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Set cover and maximum coverage problems are fundamental algorithmic 
problems that arise frequently in a variety of settings. Their importance is partly 
due to the fact that many covering problems can be reduced to these problems. 
The greedy algorithm that iteratively picks the set that covers the maximum 
number of uncovered elements is a (Inn + 1) approximation for the set cover 
problem and an approximation^ for the maximum coverage problem [15]. 
Feige [12] showed that these ratios are optimal unless NP is contained in quasi- 
polynomial time. In a number of applications the set system is implicitly defined 
and the number of sets is exponential in the number of elements. However, the 
greedy algorithm can still be applied if a polynomial time oracle that returns a 
set with good properties is available. 

In this paper, motivated by several applications, we study a variant of the 
maximum coverage problem that we call the maximum coverage problem with 
group budget constraints (MCG). We start with a simple motivating example, 
the multiple knapsack problem (MKP), and the analysis of the greedy algorithm 
for that problem in [9] . MKP is a generalization of the classical knapsack problem 
to several knapsacks: we are given n items, where item i has a size Si and a profit 
Pi] we are also given m knapsacks potentially of different capacities 6i , 62 , . . . , 6 m- 
The objective is to find a maximum profit subset of items that can be feasibly 
packed into the given set of knapsacks. We first consider the case where all the 
knapsacks have the same capacity 6. For simplicity, in the following discussion, 
we assume that we have an exact algorithm to solve the single knapsack problem. 
We can apply the following greedy algorithm: pick an unused knapsack and use 
the single knapsack algorithm to pack it optimally with a subset of items from the 
remaining items. It is easy to show, via standard set cover style arguments, that 
this algorithm gives an approximation [9]. That the greedy algorithm gives 
this ratio can also be seen via a reduction to the maximum coverage problem as 
follows. Let S be the set of all distinct subsets of the items that can be feasibly 
packed into a knapsack of size b. The MKP problem can be rephrased as a 
maximum coverage problem on this implicit exponential sized set system and 
we are required to pick m sets. The greedy algorithm that we described for the 
MKP can be seen to be the greedy algorithm for maximum coverage problem 
on the implicit set system above where the oracle is the optimal single knapsack 
algorithm. 

Now we consider instances of MKP in which the knapsack sizes could be 
different. In this case too we can define a greedy algorithm, however since the 
knapsacks are not identical, it is not a priori clear in which order to consider 
them. In [9] it is shown that irrespective of the ordering, the greedy algorithm 
results in a 2-approximation. Once again it is instructive to understand the 
implicit set system. Let Si be set of distinct subsets of items that can be feasibly 
packed into a knapsack of size bi and let S = It is clear that if the knapsacks 
are not identical, we no longer have a maximum coverage problem although 
the greedy algorithm still gives a constant factor approximation algorithm. The 

^ In this paper approximation ratios for both minimization and maximization problems 
will be greater than or equal to 1. 
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problem we have is the following: we are required to choose at most one set 
from each of the Si and cover as many elements as possible by the union of sets 
picked. 

Motivated by MKP and other problems that we consider later in this pa- 
per, we define a variant of the maximum coverage problem that provides an 
abstraction for covering problems that have groups of sets. 



Maximum Coverage with Group Budgets (MCG): We are given subsets 
Si, 52, ... , Sm of a ground set X. We are also given sets Gi, . . . , Ge, each Gi 
being a subset of {5i, . . . , 5^}. We call Gi a group. By making copies of sets, if 
necessary, we can assume that the groups Gi are disjoint from each other. We 
define two versions of the problem, the cardinality version and the cost version. 

In the cardinality version, we are given an integer k, and an integer bound ki 
for each group Gi. A solution is a subset H C {5i, . . . , 5m} such that \H\ < k 
and \H O Gi\ < ki for 1 < i < £. The objective is to find a solution such that the 
number of elements of X covered by the sets in H is maximized. In fact, we can 
assume without loss of generality that all ki are equal to 1. Otherwise, we can 
make ki copies of each group Gi . 

In the cost version, we associate a cost c{Sj) with each set Sj. Further, we 
are given a budget Bi for group Gi, 1 < i < £, and an overall budget B. A 
solution is a subset H C {5i, 52, . . . , 5m } such that the total cost of the sets in 
H is at most B. Further for any group Gi, the total cost of the sets in H nGi 
can be at most Bi. The objective is to find such a subset H to maximize the size 
of the union of sets in H. 

In many applications, m, the number of given subsets of X , is exponential 
in n and the sets are defined implicitly. In such cases we require a polynomial 
time oracle with some desirable properties. We now make this more precise. 
In the cardinality version, we assume there exists an oracle A that takes as 
input, a subset X' of X, and an index i. A{X',i) outputs a set Sj G Gi such 
that \Sj n X'l is maximized over all sets in Gi. We also work with approximate 
oracles. A is an a-approximate oracle if A{X',i) outputs a set Sj G Gi such 
that |5j n X'\ > i maxDgG- \D n X'\. For the cost version, we shall assume we 
are given an oracle B that takes as input a subset X' of X and a group index i. 

I s ■ dx' I 

B{X', i) outputs a set Sj G Gi such that ' ' is maximized - we shall assume 

that all sets in Gi have cost at most Bi. As with the cardinality case we also 
work with approximate oracles. 

We note that the maximum coverage problem is a special case of the car- 
dinality case of MCG and the budgeted maximum coverage problem [17] is a 
special case of the cost version of MCG. 

In this paper we show that a simple greedy algorithm gives a constant fac- 
tor approximation for both the cardinality and the cost versions of MCG. The 
analysis differs from the usual analysis for the maximum coverage problem and 
is based on the analysis for MKP in [9] . For the cardinality version we show that 
greedy gives a 2-approximation and our analysis is tight. For the cost version we 
obtain a 12-approximation. The greedy algorithm works in the oracle model as 
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well and in fact this is the main thrust of the paper and leads to the applications 
that we mention below. 

We note that for the cardinality version, an ^^-approximation is achievable 
if the input is given in an explicit form. This ratio is achieved by rounding an 
LP relaxation either by the pipage rounding technique of Ageev and Sviridenko 
[1] or the probabilistic rounding technique of Srinivasan [18]. 

Set Cover with Group Budgets (SCG): We define the set cover problem 
with group constraints. We only consider the cardinality version in this paper, 
it is easy to extend it to the cost case. We are given a ground set X of n 
items and a set S = {Si, S 2 , ■ ■ ■ , S'm} of subsets of X. The set S is partitioned 
into groups Gi, G 2 , . . . , Gt. The objective is to find a subset H of S such that 
all elements of X are covered by sets in H and \H (1 Gt\ is minimized. 

Note that if we have a single group containing all sets then the problem is the 
same as the set cover problem. Elkin and Kortsarz [10] seem to be the first to 
consider this problem for its applications and they call it the multiple set cover 
problem. They present an O(logn) approximation using a randomized rounding 
of a natural linear programming relaxation. Kortsarz [16] asked if there is a 
combinatorial algorithm for this problem. From the 2-approximation bound on 
Greedy we obtain a simple combinatorial (logn -I- 1) algorithm for SCG. It also 
has the advantage of working in the oracle model. 

We consider two problems for which we design the first constant factor ap- 
proximation algorithms by using reductions to instances of MCG in the oracle 
model. We describe them next. 

Multiple Depot fc- Traveling Repairmen Problem: The fc-traveling repair- 
men problem was considered by Fakcharoenphol, Harrelson, and Rao [11]. Their 
problem is the following. We are given a finite metric space on a set of nodes 
V induced by an edge weighted undirected graph G, and k not necessarily dis- 
tinct vertices si , S 2 , . . . , Sfe from V. A feasible solution to the problem is a set of k 
tours Ti, . . . ,Tk, with tour Ti starting at Si such that every vertex in V is visited 
by one of the tours. Given a feasible solution, we define the latency of a vertex 
as the time at which it gets visited by one of these tours. The objective is to 
minimize the sum of the latencies of the vertices in V . The problem models the 
case where fc-repairmen are available at the k locations (depots) and we need to 
visit all the sites that have repairs. The goal is to minimize the average waiting 
time of the sites. If fc = 1, the problem is the same as the the minimum latency 
problem for which a constant factor approximation was first given by Blum et 
al. [5]. The current best approximation ratio for the minimum latency problem 
is 3.59 [8]. In [11] a constant factor approximation algorithm is presented for the 
fc-traveling repairmen problem when all the repairmen start at the same vertex 
s, that is Si = S 2 = ■ • ■ = Sfc = s- In the same paper [11], the generalization 
of the fc-traveling repairmen problem to the case with multiple sources (depots) 
is left as an open problem. We obtain a constant factor approximation for this 
problem. We also obtain constant factor approximation algorithms for several 
generalizations as well. 
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Orienteering (or TSP) with Time Windows: The orienteering problem is 
defined as follows. We are given a metric space on a set of vertices V, a starting 
vertex s and a budget B. A feasible solution is a tour starting at s and having 
length at most B. The objective is to maximize the number of vertices covered by 
this tour. Blum et al. [7] gave the first constant factor approximation algorithm 
for this problem in general graphs, they obtained a ratio of 5 which has recently 
been improved to 3 in [6] . Previously, a 2 + e-approximation was known for the 
case when the metric was induced by points in the Euclidean plane [2]. 

In this paper we consider the more general problem where we associate a 
window with each vertex v. A vertex v can be visited only in the time 

interval [r«, dv\ - We shall say that is the release time of v and dv is the deadline 
of V. The objective, again, is to find a path that starts at s and maximizes the 
number of vertices visited, however we can only visit a vertex within its time 
window. This problem models a variety of situations when technicians or robots 
have to visit a number of different locations in a time period. The problem is 
referred to by different names in the literature including prize collecting traveling 
salesman problem with time windows and TSP with deadlines. Tsitsikilis [19] 
showed that this problem is strongly NP-complete even when the metric space is 
a line. Bar- Yehuda et al. [4] gave an 0(log n) approximation when the vertices lie 
on a line. Recently Bansal et al. [6] gave an 0(log^ n) for the general problem. In 
this paper we consider the case when the number of distinct time windows, k, is a 
fixed constant independent of the input. We give a constant factor approximation 
algorithm for this problem using a reduction to MCG and using the algorithm 
in [7, 6] as an oracle. 

The thrust of this paper is the definition of MCG and its applications. In this 
extended abstract we have not attempted to optimize the constants that can be 
achieved for the problems we consider. We defer this to the final version of the 
paper. 

2 Greedy Algorithm for MCG 

In this section we show that simple greedy algorithms give constant factor ap- 
proximation ratios for MCG. First we consider the cardinality version of the 
problem. 

2.1 Cardinality Version 

We can assume without loss of generality that the number of groups £> k. We 
work in the oracle model and assume that we have an a-approximate oracle. The 
greedy algorithm we consider is a natural generalization of the greedy algorithm 
for the maximum coverage problem. It iteratively picks sets that cover the max- 
imum number of uncovered elements, however it considers sets only from those 
groups that have not already had a set picked from them. The precise algorithm 
is stated below. 
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Algorithm Greedy 
H X' ^ X. 

For j = 1, 2, . . . , fc do 
For i= 1, . . . do 

If a set from Gi has not been added to H then Ai ^ A(Gi, X') 

Else Aj ^ 0 
EndFor 

r ^ argmaxJAil 
ff ^ HU {Ar}, X' ^ X' - Ar 
EndFor 
Output H. 

By renumbering the groups, we can assume that Greedy picks a set from 
group Gj in the jth iteration. Let OPT denote some fixed optimal solution and 
let zi < t 2 < . . . < tfc be the indices of the groups that OPT picks sets from. We set 
up a bijection tt from {1, 2, . . . , fc} to ■ ■ ■ , *fc} as follows. For 1 < h < k, 

if h € {* 1 , Z 2 , • ■ • Ofc} then we require that 7r(/i) = h. We choose tt to be any 
bijection that respects this constraint. 

Let Cj be the set that Greedy picks from Gj, and let Oj be the set that 
OPT picks from We let A' = Aj — denote the set of new elements 

that Greedy adds in the jth iteration. Let G = UjAj and O = UjOj denote the 
number of elements that Greedy and OPT pick. 

Lemma 1. For 1 < j <k, |A' | > ^\Oj — G\. 

Proof. If Oj — G = 0 there is nothing to prove. When Greedy picked Aj, the set 
Oj was available to be picked. Greedy did not pick Oj because |A' | was at least 
i|Oj — U-^^\A/j|. Since u{~\A^ C G, the lemma follows. □ 

Theorem 1. Greedy is an (a + 1)- approximation algorithm for the cardinality 
MCG with an a-approximate oracle. 

Proof. From Lemma I, we have that 

|G| = E ^ E - ^1 ^ -d U. 0,1 - |G|) > i(|0| - |G|). 

3 3 

Hence |G| > ^|0|. □ 

Corollary 1. If k = I, Greedy is an (a + 1)- approximation algorithm even if it 
is forced to pick sets from an adversarially chosen ordering of the groups. 

Proof. If k = £, the permutation tt is the identity permutation. In this case 
Lemma I holds again. □ 

Easy examples show that our analysis of the Greedy algorithm is tight. In 
fact, in the oracle model, the ratio of 2 cannot be improved. When the set system 
is available as part of the input, the problem is hard to approximate to within 
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a factor of via a reduction from the maximum coverage problem. As we 
mentioned earlier, a matching ratio can be obtained via linear programming 
[1,18]. 

A log n + 1 Approximation for SCG: We observe that the 2-approximation 
bound for MCG can be used to obtain a log n + 1 approximation for SCG as 
follows. We simply guess the optimal value A* , that is there is an optimal cover 
H* such that max^ |iJ*nGi| < A*. We then create an instance of MCG by having 
a budget of A* on each group Gi. Greedy covers at least 1/2 the elements in X. 
Iterating Greedy log n -I- 1 times results in a solution that covers all elements. In 
each iteration the number of sets added from any given group is upper bounded 
by A*. Hence, when all elements are covered, the number of sets added from any 
group is at most (logn -|- 1)A*. 

2.2 Cost Version 

We now consider the cost version of MCG. We give a greedy algorithm for this 
problem which is similar in spirit to the one for the cardinality case but differs 
in some technical details. The algorithm that we describe below may violate the 
cost bounds for the groups or the overall cost bound B. We will later show how to 
modify the output to respect these bounds. We work in the oracle model again. 
Recall that the oracle A, given a set of elements X' and an index i returns a set 
S G Gi that approximately minimizes the ratio max^gc^ ^^c{d) ^ algorithm 
is described in more detail below. We assume without loss of generality that 

b<EUb,. 

Algorithm Cost Greedy 
H X' ^ X. 

Repeat 

For i= 1,2,...,? do 
If c{H n Gi) < Bi then Ai ^ A(X', G,). 

Else Ai^ %. 

EndFor 

r ^ argmax^J^ 

H ^ H\J {Ar}'x' ^ X' - Ar. 

Until (c(ff) > B). 

Output H. 

Note that H need not obey the budget requirements. Define Hi = H D Gi. 
For a set S chosen by the algorithm, define X{S) as the extra set of elements in 
X that are covered at the time S is added to H. Define X{Hi) = Usgij^A(S'). 
Similarly, X{H) is the set of elements covered by the algorithm. Let OPT be some 
fixed optimal solution to the given problem instance. Let O be the set of sets 
chosen by opt. Define Yi = O H Gi. We call an index i good if c{H n Gi) < Bi, 
that is the algorithm did not exceed the budget for Gj. Otherwise we call i bad. 
We omit proofs of the following lemmas in this version. 
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Lemma 2. If i is bad, then \X{Hi)\ > \^ ~ 

Corollary 2. \X{H)\ > U . ^ad 
Lemma 3. \X{H)\ > UAeF.^I- 

From Corollary 2 and Lemma 3, it follows that X{H) > 2 (a+i) OPT. But H 
does not respect all the budget constraints. We partition H into three subsets 
Hi,H 2 ,H^. i?3 is the last set picked by our algorithm. H 2 contains those sets S 
which when added to H caused the budget of some group Gi to be violated ~ 
however we do not include the set in H^. Hi contains all the remaining sets in 
H. It is easy to see that Hi,H 2 and H 3 do not violate the budget constraints. 
Further, one of these three sets must be covering at least 1/3 the number of 
elements covered by H . Thus we get the following theorem. 

Theorem 2. The algorithm CostGreedy is a 6(0; + 1)- approximation algo- 
rithm for the cost version of MCG. 

3 Applications of MCG 

3.1 The fc- Traveling Repairmen Problem 

Recall from Section 1 that in the fc-traveling repairmen problem we are given a 
metric space on a set of nodes V induced by an edge weighted undirected graph 
G. We are given a set of k source vertices in V, call them si, S2 , . . . , Sfe. A feasible 
solution to the problem is a set of k tours, one tour starting at each source Si, 
such that every vertex in V is visited by one of the tours. Given a feasible 
solution, we define the latency of a vertex as the time at which it gets visited by 
one of these tours. The objective is to minimize the sum of the latencies of the 
vertices in V . 

We give the first constant factor approximation algorithm for the multiple 
depot fc-traveling repairmen problem. We define a related problem, which we 
call the budgeted cover problem, as follows. The input to the problem is a subset 
V of V and a positive integer B. A solution is a set of k tours, one tour starting 
at each source s^, such that no tour has length more than B. The objective is 
to maximize the number of vertices of V covered by these tours. We can view 
this problem in the framework of MCG as follows. The ground set is the set of 
vertices in V . For each source s,, we have a collection of sets S{,. . . ,Sf: each 
set corresponds to a distinct tour of length at most B beginning at Si . There are 
k groups, one group for each source vertex Si. The group Gi corresponding to 
Si is {S'!, . . . , S'!.}. Clearly, the cardinality version of MCG for this set system is 
the same as the budgeted cover problem for the original graph. From Section 2 
we will get a constant factor approximation algorithm for the budgeted cover 
problem provided we have the following oracle A. 

The oracle A should be able to solve the budget-MST problem. In this prob- 
lem, we are given a graph G = {V,E), & source s G C and a budget B. A solution 
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is a tour starting at s and having cost at most B. The objective is to maximize 
the number of vertices covered by the tour. Unfortunately the budget- MS T prob- 
lem is NP-hard. However, by using the algorithm for the z-MST problem [13, 
3], we can obtain a polynomial time algorithm. A, which covers OPT vertices 
and costs at most (3 ■ B. Here /? > 1 is the approximation ratio for the z-MST 
problem. Hence, if we are willing to violate the budget constraint by a factor of 
[3 we can cover as many vertices as the optimal solution. 

It follows from Theorem 1 that we can get a polynomial time algorithm for 
the budgeted cover problem which covers at least half as many vertices as the 
optimum, and constructs k tours, each tour of length at most f3B. We call this 
algorithm C. 

We can now describe our algorithm for the traveling fc-traveling repairmen 
problem. Our algorithm works in phases. We assume without loss of generality 
that all distances are at least 1. Let Vj be the set of uncovered vertices at the 
beginning of phase j (so Vq = V). In phase j, we cover as many vertices as 
possible so that the budget of each tour is about 2B More precisely, we do the 
following 

Algorithm Visit (j) : 

For p = 1, 2 do 

Run C on the budget cover problem instance with inputs Vj and 2^ . 

Remove from Vj the covered vertices. 

This describes phase j. We invoke the subroutine Visit (j) with increasing 
values of j until all vertices are covered. Given a source Si, we have constructed 
several tours starting from s^. We just stitch them together starting in the order 
these tours were found by the algorithm. Clearly, our algorithm produces a 
feasible solution. It remains to prove that it is a constant factor approximation 
algorithm. 

We begin with some notation. Fix an optimal solution opt. Let Oj denote 
the set of nodes in OPt’s solution which have latency at most 2-1 . Let Cj be the 
set of nodes visited by our algorithm by the end of phase j. 

Lemma 4. Visit(j) covers at least j\Oj — Cj-\\ vertices. 

Proof. Let Rj denote Oj — Cj-i. Let Aj be the set of nodes covered by Visit(j) 
when p = 1. Theorem 1 implies that \Aj\ > One more application of this 

theorem when p = 2 gives the desired result. □ 

The rest of the proof goes along the same lines as in [11]. Let Uj be the set 
of nodes in opt whose latency is more than 2^ . Let zz' be the set of nodes in our 
tour which do not get visited by the end of phase j. 

Lemma 5. n' < jUj. 

Proof. From Lemma 4 it is easy to see that zz' < zz'_;^ — 3/4|Oj — V,_i |. Clearly, 
\Oj \ = n — Uj and |Vj-i| = n — zz'_]^. Combining these proves the lemma. □ 
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The total latency of the tours obtained by our algorithm is upper bounded 
by Ylj '^132^'n'j and that produced by the tours in the optimal solution is lower 
bounded by 2^~^rij. From Lemma 5 we obtain that 




j j j 



which implies that 

^2-^n^- < 3^2^ ^7ij. 
j j 

This proves that our algorithm yields a 12/3 approximation. We can improve 
the ratio by using ideas from [14, 11, 8]; we defer the details to the final version. 
We can also obtain constant factor approximations for each of the following 
generalizations: (i) each vertex v can be serviced only by a given subset Sy of 
repairmen, (ii) each vertex v has a service time py that the repairmen needs to 
spend at the vertex, and (iii) each vertex v has a weight Wy and the objective is 
to minimize the sum of the weighted latencies. 



3.2 The Orienteering Problem with Time Windows 

We now consider the orienteering problem with time windows. We assume that 
the number of distinct time windows is some fixed constant k. We use, as a 
subroutine, the algorithm of Bansal et al. [6] which provides a 3-approximation 
for the case when there is a single time window [0, D] for all vertices and the tour 
is required to start at a vertex s and end at a vertex t. In the rest of the section 
we use [3 to denote the approximation ratio for the single deadline case. All our 
ratios will be expressed as functions of j3. Let A be the maximum distance in the 
metric space. We begin by describing approximation algorithms for two special 
cases: (1) when all release times are zero, and (2) when all deadlines are the 
same. 

Release Times Are Zero: We consider the special case when = 0 for all 
nodes v. Let d\ < d 2 <■■■< dk he the k distinct deadlines. Let Vi denote the 
set of vertices whose deadline is di. Let P* be the tour constructed by some 
optimal solution. Define uj as the source vertex s. For 1 < / < fc, let w* as the 
last vertex in the tour P* which is visited by the deadline di . It is possible that 
V* = V*, for two distinct indices i and i' . Suppose v* is visited at time t*, then 
it follows that that t* < di. 

Our algorithm first guesses the vertices w*, . . . , and the time instances 

t\,t 2 ,..., tl- Note that t* < nA. Hence the total number of guesses is 0{n^^A^). 
Since A need not be polynomially bounded, the number of guesses is not poly- 
bounded. We omit details on how to use a polynomial number of guesses. Now, 
we define k groups Gi,. . . ,Gk as follows. Gi is the set of all paths on the vertex 
set Vi U Vj+i U • • • U Vfe which originate at v*_i and end at v* with the additional 
constraint that the length of the path is at most t* — t*_i- 
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Lemma 6. Consider the instance of the MCG with groups defined as above 
where we need to pick exactly one set from each group. A ^ -approximation al- 
gorithm for this problem instance implies the same for the corresponding orien- 
teering problem. 

Proof. Suppose we are given a solution to the MCG which picks paths Pi,. .. ,Pk 
from the corresponding groups. If we stitch these tours sequentially, it is easy 
to see that we get a path which satisfies the deadlines of the vertices visited by 
the individual paths. Therefore the number of vertices covered by this tour is 
|Pi U P2 U • • • U Pfe|. Further, if we consider the tour P*, we can get a solution to 
the MCG which covers |P*| vertices. This proves the lemma. □ 

Thus, it is enough to approximate the MCG induced by the guess of w*, . . . , 
and t*, . . . ,t^. The oracle needed for this instance is an algorithm for the ori- 
enteering problem where the time windows for all the nodes are of the form 
[0,Z?], hence we can use the algorithm of Blum et al. [7] or Bansal et al. [6]. 
From Corollary 1 we obtain a a (/? -I- l)-approximation algorithm for the case 
of k deadlines. The running time of the algorithm is 0{n^^ A^T) where T is the 
running time of the approximation algorithm for the single deadline case. 

Single Deadline: We now consider the special case when all deadlines are the 
same, say D but the release dates can be different. Consider any feasible tour P 
which starts at s and ends at a vertex u. Let the length of P be i{P). Suppose we 
reverse the tour, i.e., we view the tour as a path P’’ starting at u and ending at s. 
If P visits a vertex v at time t, then P'" visits v at time £{P)—t. So r^ < t < £{P) 
implies that 0 < £{P) — t < i{P) — r„. Thus, we can view this tour as one in 
which the release time of a vertex is 0 and the deadline is £(P) — r„. Therefore, if 
we could guess the length of the optimal path P* and the last vertex u* in this 
path, then we could just use the algorithm mentioned in the previous section. 
Thus, we can get a (/? -I- l)-approximation algorithm for this problem as well. 
The running time of the algorithm increases by a factor of Z\ from the algorithm 
for single release date. 

k Time Windows: Now we address the case where there are both release times 
and deadlines. Let ri < r^ < . . . < be the k distinct release time and let 
di < d 2 <...< dk he the k distinct deadlines. Let P* be the tour constructed 
by the optimal solution. As before let v* be the last vertex in P* to be visited 
before di and let t* be the time at which v* is visited. Recall that Vj is the 
set of vertices with deadline di. We define group Gi to be the set of tours that 
start at v*_i at t*_i and end at v* by t*. The vertices that a tour in Gi can 
visit are constrained to be in Vi U Vi+i U . . . U 14. Lemma 6 trivially generalizes 
to this setting to yield a 7 approximation provided we have an oracle for the 
MCG instance. Consider a group Gi. The vertices all have a deadline at least as 
large as t*, hence we have a single deadline. The vertices might have different 
release times, however there are at most k distinct release times. Hence the oracle 
needed for Gi can be obtained from the algorithm described above for this case 
that has an approximation ratio of (3 -\-l. Thus, once again applying Theorem 1 
we can obtain a (/? -I- 2)-approximation algorithm for the case of k time windows. 
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Theorem 3. Given a j3- approximation algorithm for the orienteering problem 
with a single time window for all vertices, there is a {(3 + 2) -approximation al- 
gorithm for the orienteering problem with at most k distinct time windows that 
runs in time polynomial in {nA)^ . 
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Abstract. In the Minimum Common String Partition problem (MCSP) 
we are given two strings on input, and we wish to partition them into the 
same collection of substrings, minimimizing the number of the substrings 
in the partition. Even a special case, denoted 2-MCSP, where each let- 
ter occurs at most twice in each input string, is NP-hard. We study a 
greedy algorithm for MCSP that at each step extracts a longest common 
substring from the given strings. We show that the approximation ratio 
of this algorithm is between and 0(n° ®®). In case of 2-MCSP, 

we show that the approximation ratio is equal to 3. For 4-MCSP, we give 
a lower bound of 17(logn). 

1 Introduction 

By a partition of a string A we mean a sequence V = (Pi, P2, . . . , Pm) of strings 
whose concatenation is equal to A, that is P1P2 . . . Pm = A. The strings Pi are 
called the blocks of P. If P is a partition of A and Q is a partition of B, then 
the pair tt = (P, Q) is called a common partition of A, B, if Q is a permutation 
of P. For example, tt = {{ab,bccad,cab), {bccad,cab,ab)) is a common partition 
of strings A = abbccadcab and B = bccadcabab. 

The minimum common string partition problem (MCSP) is defined as follows: 
given two strings A, B, find a common partition of A, B with the minimal number 
of blocks, or report that no common partition exists. By fc-MCSP we denote the 
version of MCSP where each letter occurs at most k times in each input string. 

The necessary and sufficient condition for A, B to have a common partition 
is that each letter has the same number of occurrences in A and B. Strings with 
this property are called related. Verifying whether two strings are related can be 
done easily in linear time, and for the rest of the paper we assume, without loss 
of generality, that the input strings are related. 

In this article, we deal with approximations of MCSP. In particular, we study 
the greedy algorithm for MCSP that constructs a common partition by iteratively 
extracting the longest common substring of the input strings. More precisely, the 
algorithm can be described in pseudo-code as follows: 
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Algorithm Greedy 

Let A and B be two related input strings 
while there are symbols in A or B outside marked blocks do 
S ^ longest common substring of A, B that does not 
overlap previously marked blocks 
mark one occurrence of S in each of A and B as blocks 

sequence of consecutive blocks in A and B, respectively 

For example, if A = cdabcdabceab, B = abceabcdabcd, then Greedy first 
marks substring abcdabc, then ab, and then three single-letter substrings c, d, e, so 
the resulting partition is ((c, d, abcdabc, e, ab), {ab, c, e, abcdabc, d)), while the op- 
timal partition is {{cdabcd, abceab), {abceab, cdabcd)). As illustrated by the above 
example, the common partition computed by Greedy is not necessarily optimal. 
The question we study is what is the approximation ratio of Greedy on MCSP 
and its variants. We prove the following results (by n we denote the length of 
the input strings): 

Theorem 1.1. (a) The approximation ratio of Greedy for MCSP is between 

12 (^ 0 . 43 ) 

(b) For 4-MCSP, the approximation ratio o/ G reedy is at least l7(logn). 

(c) For 2-MCSP, the approximation ratio o/ G reedy is equal to 3. 

Our results can be extended to the variation of the minimum common parti- 
tion problem, where letters have signs associated with them (cf. [1]). A substring 
P may now appear in A either as P or as its reversal P^, where in the rever- 
sal the signs of all letters are reversed as well. As in MCSP, we want to find a 
minimum common partition of A and B into such substrings. 

Related Work. The minimum common string partition problem was introduced 
by Chen et al. [1]. They pointed out that MCSP is closely related to the well- 
known problem of sorting by reversals and they use MCSP for comparison of 
two DNA sequences. In this application, the letters in the alphabet represent 
different genes in the DNA sequences, and the cardinality of the minimum com- 
mon partition measures the similarity of these sequences. The restricted case of 
/c-MCSP is of particular interest here. Goldstein et al. [3] proved that 2-MCSP 
is NP-hard. 

The size of the minimum partition of A and B can be thought of as a distance 
between A and B. The classical edit-distance of two strings is defined as the 
smallest number of insertions, deletions, and substitutions required to change one 
string into another [5]. Kruskal and Sankoff [4], and Tichy [8] started to consider 
block operations in string comparison, in addition to the character operations. 
Lopresti and Tomkins [6] investigated several different distance meassures; one 
of them is identical to the MCSP measure. 

Shapira and Storer [7] study the problem of edit distance with moves in 
which the allowed string operations are the following: insert a character, delete 
a character, move a substring. They observe that if the input strings A, B are 
related, then the minimum number of the above listed operations needed to 
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convert A into B is within a constant factor of the minimum number of only 
substring movements needed to convert A into B; the later is within a constant 
factor of the minimum common partition size. Shapira and Storer also considered 
a greedy algorithm nearly identical to ours and claimed an O(logn) upper bound 
on its approximation ratio; as it turns out, however, the analysis is flawed. 

Cormode and Muthukrishnan [2] describe an 0(lognlog* n)-approximation 
algorithm for the problem of edit distance with moves. As explained above, 
this result yields an 0(lognlog* n)-approximation for MCSP. Better bounds for 
MCSP are known for some special cases. A 1.5-approximation algorithm for 
2-MCSP was given by Chen et al. [1]; a 1.1037-approximation algorithm for 2- 
MCSP and a 4-approximation algorithm for 3- MCSP were given by Goldstein et 
al. [3]. All these algorithms are considerably more complicated than Greedy. 
Due to its simplicity and ease of implementation. Greedy is a likely choice for 
solving MCSP in many practical situations, and thus its analysis is of its own 
independent interest. 

2 Preliminaries 

By A = ai 02 . . . a„ and B = b\b 2 ■ ■ - bn we denote the two arbitrary, but fixed, 
input strings of Greedy. W.l.o.g., we assume that A and B are related. If tt is 
a common partition of A, B, then we use notation #blocks{Tr) for the number of 
blocks in tt, that we refer to as the size of tt. The size of a minimum partition 
of A, B is denoted by dist{A,B). 

We typically deal with occurrences of letters in strings, rather than with 
letters themselves. By a “substring” we mean (unless stated otherwise) a spe- 
cific occurrence of one string in another. Thus we identify a substring S = 
OpOp+i . . . Op+s of A with the set of indices {p,p -I- 1, . . . ,p -I- s} and we write 
S = {p,p + I, ■ ■ ■ ,p + s}, where [S'] = s -k 1 is the length of S. Of course, the 
same convention applies to substrings of B. If S' is a common substring of A, B, 
we use notations and S® to distinguish between the occurrences of S in A 
and B. 

Partitions as Functions. Suppose that we are given a bijection ^ : [n] ^ [n] 
(where [n] = {1,2, . . . ,n}) that preserves letters of A and B, that is, = Ui 
for all i G [n]. A pair of consecutive positions i,z -k 1 G [n] is called a break of 
C if ^{i + 1) yf ^(z) + 1. Let #breaks{^) denote the number of breaks in For a 
common substring S of A, B, say S = OpOp+i . . . Up+s = bqbg+i . . . bq+s, we say 
that ^ respects S if it maps consecutive letters of onto consecutive letters in 
S^, that is, ^(z) = z-kq— pforzG 

A letter-preserving bijection ^ induces a common partition (also denoted 
for simplicity) whose blocks are the maximum length substrings of A that 
do not contain breaks of The partition obtained in this way does not have 
any “unnecessary” blocks, i.e., #blocks{^) = #breaks{^) + 1. And vice versa, if 
TT = {V, Q) is a common partition of A, B, we can think of tt as a letter-preserving 
bijection tt : [rz] — > [n] that respects each block of the partition. Obviously, we 
then have #blocks{n) > #breaks{Tr) -k 1. We use this relationship throughout the 
paper, identifying common partitions with their corresponding functions. 
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Reference Partitions. Let tt be a minimum common partition of A and B. 
(This partition may not be unique, but for all A, B, we choose one minimum 
common partition in some arbitrary way.) In the first step, Greedy is guaran- 
teed to find a substring Si of length at least the maximum length of a block in 
TT. For the analysis of Greedy, we would like to have a similar estimate for all 
later steps, too. However, already in the second step there is no guarantee that 
Greedy finds a substring as long as the second longest block in tt, since this 
block might overlap Si and it may be now partially marked (in A or B). To get 
a lower estimate on j^tl, for t > 1, the idea is introduce a corresponding refer- 
ence common partition of A, B that respects all the blocks Si, , St-i selected 
by Greedy in steps 1 to t — 1. This partition may gradually “deteriorate” (in 
comparison to the minimum partition of A and B), that is, it may include more 
blocks and its blocks may get shorter. Furthermore, it may not be a minimum 
common partition of the unmarked segments. It is only used to estimate the 
“damage” caused by Greedy when it makes wrong choices (that is, when it 
marks strings which are not in the optimum partition). 

Denote by g the number of steps of Greedy on A, B. For t = 0,1, ... ,g, 
the reference common partition pt is defined inductively as follows. Initially, 
po = TT. Consider any t = 1,. . . ,g. Suppose that Sf = {p,p 1, . . . ,p s} and 
Sf^ = {q,q-\- 1, ... ,q-\- s}. Define function 6 : Sf ^ Sf^ such that S(i) = i-hq—p 
for j € S^. Then pt is defined by 

r\ — J ^ m 

- \pt-i{S-^Pt-iY^^H{} for i€[n]~ SY 

where £{i) = min {? > 0 : pt-i{S~^ pt-iY{i) Y- We now show that each pt 
is well-defined and bound the increase of the number of breaks from pt-i to pp. 

Lemma 2.1. For each t = 1, . . . ,g, (a) pt is a common partition of A, B, (b) 
Pt respects Si, . . . ,St, (c) #breaks{pt) < #breaks{pt-i) + 4. 

Proof. The proof of the lemma is by induction. For t = 0, (a) and (b) are trivially 
true. Suppose that t > 0 and that the lemma holds for t—1. To simplify notation 
let S = St, p = pt-i and p' = pt. 

Consider a bipartite graph G C [n] x [n], with edges {i,p{i)), for z G [n], and 
(z,(5(z)), for i G S^. These two types of edges are called p-edges and 5-edges, 
respectively. 

Let S^ = [n] — S^ and S^ = [n] — S^ . In this proof, to avoid introducing 
additional notation, we think of S^ and S'^ as the sets of nodes on the “left- 
hand” side of G and S^ and S^ as the nodes on the “right-hand” side. Then, 
any node in S'^ or S^ is incident to one p-edge, and each node in S^ or S^ is 
incident to one p-edge and one (5-edge. Thus, G is a collection of vertex disjoint 
paths and cycles whose edges alternate between p-edges and 5-edges. We call 
them G-paths and G-cycles. All G-cycles have even length and contain only 
nodes from S^ and S^ . All maximal G-paths have odd lengths, start in S^, 
end in S^ , and their interior vertices are in S^ or S'®. The G-path starting at i 
has the form i, p{i), p{i), p5~^ p{i), . . . , p{5~^ pY^’‘Yi). Thus, for i G S^, p'(z) 
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cabccclauaabahccubabdcd 




bdcdcabccababdaaaccaba 
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Fig. 1. An example illustrating the construction of p' . The left part shows p and some 
G-paths. The right part shows p' . The strings in the partitions are numbered, and the 
common substring St = abccababd is shaded. 



is simply the other endpoint of the G-path that starts at i. This implies that p 
is 1-1 and letter-preserving, so it is indeed a common partition. Condition (b) 
follows immediately from the inductive assumption and the definition of p' . It 
remains to prove (c). 

Lemma 2.2. Suppose that i,i + 1 is a break of p' . Then there is I such that 
I < Toim {^{i) , l{i + 1)} and one of the following conditions holds: 

(BO) Exactly one of i,i + l is in S^. 

(Bl) i,i + 1 G and {5~^pY{i+ 1) is a break of p. 

(B2) i,i + 1 € and exactly one of p{6~^ pY (i) , p{5~^ pY{i + 1) belongs to . 

We refer to breaks (BO), (Bl), (B2), respectively, as breaks induced by the 
endpoints of S^, breaks induced by the breaks inside (only if i,i -|- 1 is a new 
break), and breaks induced by the endpoints of S'®. 

Proof. If exactly one of f, f -I- 1 is in the case (BO) holds. Since z, i -I- 1 is 
never a break in p' if both i and z -I- 1 are in S^, we assume that z, z -I- 1 G 
for the rest of the proof. 

Consider the largest integer I < min {£(z), ^(z -b 1)} for which {S~^pY{i), 
{S~^pY(i + 1) are consecutive in S^, that is {S~^pY{i -b 1) = {6~^pY{i) + 1. 
Let j = {5~^pY{i). We have two sub-cases. If p{j -b 1) yf p{j) + 1, then j,j -b 1 is 
a break of p, so condition (Bl) is satisfied. If p{j -b 1) = p{j) + 1, then at least 
one of p{j), p{j -b 1) must be in S® , for otherwise z, z -b 1 would not be a break 
of p' . But we also cannot have both p{j), p{j -b 1) G S®, since then (i5“^p)*+^(z), 
(<5“^p)^“''^(z -b 1) would be consecutive in S^, violating the choice of 1. Therefore 
case (B2) holds. □ 

We now complete the proof of part (c) of Lemma 2.1. There are no breaks 
of p' inside and we have at most two breaks of type (BO) corresponding 
to the endpoints of S^. By the disjointness of G-paths and cycles, there are at 
most two breaks of p' of type (B2), each corresponding to one endpoint of S'®. 
Similarly, each break of p (inside or outside S^) induces at most one break of p' 
of type (Bl). This implies (c), and the proof of the lemma is now complete. □ 

Note that we did not use the fact that S has maximum length. So our con- 
struction of pt can be used to convert any common partition tt into another 
partition tt' that respects a given common substring S, and has at most four 
more breaks than tt. 

Lemma 2.1 implies that in every step t, every block in pt is either completely 
marked or unmarked. 






The Greedy Algorithm for the Minimum Common String Partition Problem 



89 



3 Upper Bound for MCSP 

In this section we show that Greedy’s approximation ratio is The 

proof uses reference common partitions introduced in Section 2 to keep track of 
the length of the common substrings selected by Greedy. 

For p > <7 > 1 , we define H (p, q) to be the smallest number h with the 
following property: for any input strings A, B, if at some step t of Greedy there 
are at most p unmarked symbols in A and at most q unmarked blocks in the 
current reference partition pt, then Greedy makes at most h more steps until it 
stops (so its final common partition has at most t+h blocks.) For convenience, we 
allow non-integral p and q in the definition. Note that H{p, q) is non-decreasing 
in both variables. 

Before proving the upper bound, note that Lemma 2.1 immediately 

gives a recurrence H{p,q) < H{p{l — 1/g), g -|- 3) -I- 1 (whenever both values of 
H are defined), as in one step of Greedy, the longest common substring has at 
least 2 letters (which will be marked in the next partition), and the number of 
unmarked blocks in the reference partition increases by at most 3. With some 
calculation, this gives a slightly weaker but simpler bound of 0(n°’^^). 

The idea of the proof of the improved bound is to consider, instead of one 
step of Greedy, a number of steps proportional to the number of blocks in 
the original optimal partition, and show that during these steps Greedy marks 
a constant fraction of the input string. This yields an improved recurrence for 
H{p, q). 

Lemma 3.1. For all p, q satisfying p > 9q/5 + 3, we have 
H{p, q) < H{5p/6, (3g + 5)/2) + {q + 5)/6. 

Proof. Consider a computation of Greedy on A, B, where, at some time t (i.e., 
after t blocks have already been marked), there are p unmarked symbols in A, and 
q unmarked reference blocks of pt- We denote these blocks by Ri, R 2 , ■ ■ ■ , Rq, in 
the order of non-increasing length, that is \Rz\ > |i?z+i|, for z = 1, . . . , g — 1. We 
analyze the computation of Greedy starting at time t. Let g be the number of 
additional steps that Greedy makes. Our goal is to show that g < H{5p/6, (3g-l- 
5)/2) -I- (g -I- 5)/6. (Since the bound is monotone in p and g, we do not need to 
consider the case of fewer than g unmarked blocks or fewer than p unmarked 
symbols.) If p < (g -I- 5)/6, this trivially holds, so in the rest of the proof we 
assume that g > (g -I- 5)/6. 

Let Ti = St+i be the common substring selected by Greedy in step t + i. 
We say that Greedy hits R^ in step t + i if Ti overlaps Rz, either in A or in B, 
that is, if either n R^ yf 0 or T® n yf 0. 

Claim A. For all j = 1, . . . , g, the total length of the blocks Ri, . . . ,Rg that are 
hit by Greedy in A in steps t + 1, . . . ,t + j is at most 6 J2i=i |U|- 

We estimate the total length of the blocks Rz that are hit at step t + ivaA 
but have not been hit in steps — 1. The total length of the blocks 

that are contained in Sf and Sf is at most 2|Ti|. There are up to four blocks 
that are hit partially, but by the greedy choice of R, each has length at most 
I Til, and the claim follows. □ 
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Claim B. l^il > p . 

Let i be the minimum integer such that 6 X)i=i 1^*1 — V- Since X)i=i 1^*1 = Pj 
I is well defined and i < g- For j = 1, . . . define \j as the maximal index 

for which \Rx\ < l^il- Since & |F>| < p = \Rx\, all Xj 

are well defined, and Xe < q. We also note that Ai = 0. For each j = 

Claim A implies that one of the blocks i?i, . . . ,R\.+i is not hit by any of the 
blocks Ti, . . . ,Tj-i and thus, by the definition of Greedy and the ordering of 
the blocks Rz, \Tj\ > Considering again the ordering of the blocks Rz, 

we have 6|Tj| > I-Ra^+iI + + l-RA^+el- We conclude that Aj+i > Xj + 6, for 

j = 1, 1. This, in turn, implies that g > A^ + 1 > 6^ — 5. So ^ < (g + 5)/6, 

and Claim B follows, by the choice of ^ and its integrality. □ 

By Claim B, after exactly [(g + 5)/6j steps. Greedy marks at least p/6 
letters, so the number of remaining unmarked letters is at most p' = 5p/6. By 
Lemma 2.1, the number of unmarked blocks increases by at most 3 in each step 
(since one new block is marked), so the number of unmarked blocks induced by 
Greedy in these [(g + 5)/6j steps is at most 3[{q + 5)/6j < (g + 5)/2. Thus the 
total number of unmarked blocks after these steps is at most q' = q+ {q + 5)/2 = 
{3q + 5) /2. The condition in the lemma guarantees that H{p', q') is defined, so, 
by induction, the total number of steps is at most H{p' , q') + {q + 5) /6. □ 

Finally, we prove the upper bound in Theorem 1.1(a). We prove by induction 
on p that for p > <7 and a sufficiently large constant C, H (p, q) < Cp~* {q + 
5)^“'’' — ^q. We choose C so that for all g < p < 9q/5 + 3, the right-hand side 
is at least p and thus the inequality is valid. For p > 9q/5 + 3, by Lemma 3.1, 
the inductive assumption, and the choice of 7, we have H{p, q) < H{^p, + 

^ < Cilpnliq + b))^-'^ _ 1 . 3^ + = Cp'^(g + 5)1-7 _ 1^. Let A, B be 

input strings of with length n and dist{A, B) = m. Then the number of blocks 
in Greedy’s partition is at most H{n,m) = 0{nX)m, and the theorem follows. 

4 Lower Bound for MCSP 

We show that the approximation ratio of Greedy is 

We first construct strings Ci, Di, Ei, Fi as follows. Initially, Cq = a and Dq = b. 
Suppose we already have Ci and Di, and let Ei be the set of letters used in Ci, 
Di. Create a new alphabet E[ that for each a G E has a new letter, say, a'. 
We first create strings Ei and Fi by replacing all letters a G E in Ci and Di, 
respectively, by their corresponding letters a! G E[. Then, let 

Gi+i = CiD^E^DiCi, and A+i = D^EiF^EiD^. 

For each i, we consider the instance of strings Ai = CiDi and Bi = DiCi. 
For example. Eg = c, Fq = d, Aq = ab, Bq = ba, C\ = abcba, Di = bcdcb, 
Ai = abcbabcdcb, and B\ = bcdcbabcba, etc. 

Let n = 2 • 5*. We have \Ai\ = \Bi\ = n and dist{Ai, Bi) < 2. We claim that 
Greedy’s common partition of Ai and Bi has 2*+^ — 2 = *“82 5) substrings. 

We assume here that Greedy does not specify how the ties are broken, that is. 
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whenever a longest substring can be chosen in two or more different ways, we 
can decide which choice Greedy makes. 

The proof is by induction. For z = 0, Greedy produces two substrings, as 
claimed. For z > 0, 



^i+i — Ci Di Ei Di Ci Di Ei Fi Ei Di, Bi^i — Di Ei Fi Ei Di Ci Di Ei Di Ci. 

There are three common substrings of length CiDiEiDiCi, DiEiFiEiDi, 

and EiDiCiDiEi, and no longer common substrings exist. To justify this, we 
use the fact that the alphabet of Ci, Di is disjoint from the alphabet of Ei, 
Fi- Suppose that S' is a common substring of length at least 5*+^. To have this 
length, S must contain either the first or the second Ei from Ai+i. We now have 
some cases depending on which Ei is contained in S, and where it is mapped 
into Bi^i via the occurrence of S in If S contains the first Ei, then, by the 
assumption about the alphabets, this Ei must be mapped into either EiFiEi or 
into the last Ei in Bz+i . If it is mapped into EiFiEi, then S must be EiDiCiDiEi. 
If it is mapped into the last Ei in then S must be CiDiEiDiCi. In the last 
case, S contains the second Ei. By the same considerations as in the first case, 
it is easy to show that then S must be either DiEiFiEiDi or EiDiCiDiEi. 

Breaking the tie, assume that Greedy marks substring EiDiCiDiEi. The 
modified strings are: 



Gj Di Ei A Gj Di Ei Fj Ei Di, 



Di Ei El Ei Di Cl Di Ei Di Ci, 



where the overline indicates the marked substring. In the first string the un- 
marked segments are Ai, A[Di, and in the second string the unmarked segments 
are Bi and DiB[, where A' = FiEi and F' = EiFi are identical as Ai, Bi respec- 
tively, but with the letters renamed. The argument in the previous paragraph 
and the disjointness of the alphabets implies that the maximum length of a non- 
mar ked common substring is now 5*. We break the tie again, and make Greedy 
match the two Di's in FiEiDi and DiEiFi, leaving us with non-mar ked pairs 
of substrings {Ai,Bi} and By induction. Greedy produces 2 *+^ — 2 

substrings from Ai, Bi, and the same number from A' and B[. So we get the 
total of 2 ( 2 *+^ — 2 ) -I- 2 = 2 *+^ — 2 strings. 



5 Lower Bound for Greedy on 4-MCSP 

In this section we show that Greedy’s approximation ratio is I2(logzz) even on 
4-MCSP instances. To simplify the description, we allow the input instances A, B 
to be multisets of equal number of strings, rather than single strings. It is quite 
easy to see that this does not significantly affect the performance of Greedy, for 
we can always replace A, B by two strings A, B, as follows: li A= {Ai ,... , Am} 
and B = {Bi , . . . ,F„}, let A = AixijjiA 2 X 2 y 2 ■ . . A„i-iXm-iym-iA„i and B = 
BiyiXiB2y2X2 ■ ■ . Bm-iym-iXm-iBm, where Xi,yi, . . . ,Xm-i,ym-i are new let- 
ters. Then both the optimal partition and the partition produced by Greedy on 
A,B are the same as on A,B, except for the singletons x\,yi, . . . ,Xm-i,ym-i- 
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Since in our construction m is a constant, it is sufficient to show a lower bound 
of l7(logn) for multisets of m strings. 

For i = 1,2,..., we fix strings qi,q[,Vi, r' that we will refer to as elementary 
strings. Each elementary string qi,q[,ri,r[ has length 3*“^ and consists of 3*“^ 
distinct and unique letters (that do not appear in any other elementary string.) 

We now recursively construct instances A'', B'‘ of 4-MCSP. The invariant of 
the construction is that have the form: 

A^: Piqi, P2qtri, Psqi, Piqir[, P^q',, Peq^Ti, Prq^, P%q[r'i 
Sh Piqir^, P2q^, P^qm^, P^qi, Pzq^n, Pe,q[, Prq'A'i, P&q[ 

where Pi, ... ,Ps are some strings of length smaller than 3®“^ with letters distinct 
from qi,q[,ri,r[. 

Initially, we set dl\ P\, ... ,P^ = e, and construct A'^, B^ as described above. 
In this case qi,q[,ri,r[ are unique, single letters. 

To construct ,8*+^, we append pairs of elementary strings to the strings 

from A^, 8*. For convenience, we omit the subscripts for q and r, writing q = qi, 
q = qi+i, etc. After rearranging the strings, the new instance is 

: Piqr q, P^qr' qf, Prq'r' q, Peq'r qf' , P^qr' q' , P2qr q'f, Psq'r q' , Pgq'r' q' f' 

: P\qr qf, P^qr' q, Prq'r' qf', Peq'r q, P^qr' q'f, P^qr q' , Psq'r q'f', Pgq'r' q' 

Note that this instance has the same structure as the previous one, since we 
can take P[ = P\qr, P 2 = Piqr', etc., thus we can continue this construction 
recursively. Each letter appears at most four times in A' and 8*, so this is indeed 
an instance of 4-MCSP; the claimed bound on the length of Pj also follows easily. 

Consider now the f-th instance, A' and 8b To estimate the optimal partition, 
we can match the 8 pairs of strings as aligned above, adding the shorter string 
from each pair to the common partition, leaving us with additional 4 strings r, 
r', f, f', so dist{A’’ , B'') < 12. 

We now show that Greedy computes a partition with 0{i) = 0(logn) 
blocks. To this end, we claim that, starting from Greedy first 

matches all suffixes that consist of two elementary strings as shown below 
and are rearranged to show the matched strings aligned vertically): 

Piqr' qf, Peq'rqf', P2qrq'f,Pgq'r'q'f', Piq rq, Prq' r'q, Psq r' q' , P^q' rq' 
Piqr qf, Prq'r' qf' , Psqr' q'f , Psq'r q'f' , Peq' rq, Piq r'q, Pgq' r'q' , P2qrq' 

Indeed, the instance has four common substrings of length 2-3*, namely qf, 
qf', q'f, q'f' , and, by the choices of the lengths of elementary strings and the 
bound on the lengths of the P/’s, all other common substrings are shorter. Thus 
Greedy starts by removing (marking) these four suffixes. Similarly, at this step, 
the new instance will have four common substrings of length 3* -I- 3*“^, namely 
rq, r'q, r'q' , rq' , and all other common substrings are shorter. So now Greedy 
will remove these four suffixes. The resulting instance is simply A', 8*, so we 
can continue recursively, getting 0{i) blocks. If n is the length (total number of 
characters) of we have i = 0{logn), so the of the lower bound is complete. 
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6 Upper Bound for Greedy on 2-MCSP Instances 

Consider two arbitrary, but fixed, related strings A = 0102 •••a„ and B = 
6162 • • • 6 n in which each letter appears at most twice. Let tt be a minimum 
common partition of A,B, and denote by g the number of steps of Greedy on 
A, B. For each t = 0, . . . , g, let pt be the common reference partition of A, B at 
step t, as defined in Section 2. In particular, po = and pg is the is the final 
partition computed by Greedy. 

Our proof that Greedy’s approximation ratio on 2-MCSP instances is at 
most 3 is based on amortized analysis. We show how to define a potential <I>t of 
Pt that has the following three properties: 

(PI) <pQ <3 ■ #blocks{po) + 1, 

(P2) <Pt < <Pt-i for t = 1, . . . , p, and 
(P3) <Pg > #blocks{pg) + 1. 

If such ^t’s exist, then, using the optimality of po and conditions (PI), (P2), 
(P3), we obtain #blocks{pg) < — 1 < < 3- #blocks{po) = 3- #blocks(Tr) = 

3 • dist{A, B), and the 3-approximation of Greedy follows immediately. Thus it 
remains to define the potential and show that it has the desired properties. 
Classification of Breaks. Consider some step t. A break i,i + ^ oi pt is called 
original if it is also a break of tt; otherwise we call this break induced. Letters 
inside blocks marked by Greedy are called marked. For any letter at in A, we 
say that at is unique in pt if at is not marked, and there is no other non-mar ked 
appearance of at in A. 

Suppose that z,z -I- 1 is an original break in pt. We say that this break is 
left-mature (resp. right-mature) if at (resp. a^+i) is unique; otherwise it is called 
left-immature (resp. right-immature). If a break is both left- and right-mature, 
we call it mature. If it is neither, we call it immature. (The intuition behind these 
terms is that, if, say, a break z,z -|- 1 is left-mature, then the value of pt{i) does 
not change anymore as t grows.) We extend this terminology to the endpoints of 
A. For the left endpoint, if oi is unique in pt we call this endpoint right-mature, 
otherwise it is immature. Analogous definitions apply to the right endpoint. 
Potential. We first assign potentials to the breaks and endpoints of pp. 

((^1) Each induced break has potential 1. 

(</)2) The potential of an original break depends on the degree of maturity. If a 
break is immature it has potential 3. If it is left-mature and right-immature, 
or vice versa, it has potential 2. If it is mature, it has potential 1. 

{<f>3) The left endpoint has potential 2 or 1, depending on whether it is right- 
immature or right-mature, respectively. The potential of the right endpoint 
is defined in a symmetric fashion. 

Then <Pt is defined as the sum of the potentials of the breaks and endpoints 
in Pt. For t = 0, all breaks in tt have potential at most 3 and the endpoints have 
potential at most 2, so we have <Pq < 3 - #breaks{Tr) -|- 4 = 3 • #blocks{TT) -\- 1. For 
t = g, all letters in A are marked, so the potentials of all breaks and endpoints 
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are equal 1. Therefore <Pg > #breaks{pg) + 2 = #blocks{pg) + 1. Thus properties 
(PI) and (P3) hold. 

To prove property (P2), first note that the potentials of the breaks of pt-i 
that remain in pt cannot increase. Thus only the new breaks (those in pt but 
not in pt-i) can contribute to the increase of the potential. All these new breaks 
have potential 1 in . With each new break we can associate some old break (or 
an endpoint) of pt-i that either disappears in pt or whose potential decreases. 
This mapping is not necessarily one-to-one. However, the number of new breaks 
associated with each old break does not exceed the decrease of the potential 
of this old break. This needs some case analysis along the lines of Lemma 2.2; 
details are omitted in this abstract. Summarizing, we obtain the the upper bound 
of 3 for 2-MCSP, i.e., the upper bound in Theorem 1.1(c). 

7 Lower Bound for Greedy on 2-MCSP 

Let I be a large even integer. Let A! = ai 02 ■ ■ - ap, B' = 6162 ■ ■ - bp, and 

= o,i{i-i)+20^i{i-i)+3 ■ ■ ■ 0^1^ ■■■ 0,1+10^1+2 ■■■ d2(i-i)+i 0203... ai ai 

B" = bp 5 /(;_i) + i6;(/_i)+2 . . . 6;2_i ... hbl+i . . . b2(l-l) 6162... 6;-l, 

where the P letters Oj of A are all distinct, for alH = 1 (mod ? -k 1), bi are new 
letters distinct from each other and from all o^', and bi = at for all other values 
of i. The instance is A = A'B” and B = A" B' . Obviously, no letter occurs more 
than twice, so this is indeed an instance of 2-MCSP. 

A' and B' consist of the same I — 1 substrings of length I, separated (and 
ended) by I distinct markers. The strings A!' and B” are obtained from A! and 
B' by cutting into I + 1 substrings of length I — 1 and a single letter, and taking 
these in the reverse order. (Note that the cuts in the two strings are not aligned.) 

The cuts indicated by spaces in the definitions of A!' and B" show that 
dist{A, B) < dist{A', A") + dist{B' , B") <2/4-4. 

A' and B' have I — 1 common substrings of length /, namely the substrings 
between the marker symbols. We claim that A and B have no other common 
substrings of length 1 . Clearly, by the choice of the markers. A! and B' have no 
other common substrings of length 1 . The longest common substring of A! and 
A!' as well as of B' and B" has length / — 1, by the definition of A!' and B" . 
The strings A" and B" also have no common string of length I or larger, since 
the ends of their substrings of length I — 1 are not aligned. There is no common 
substring that would overlap both A' and B” or A" and B'. 

Consequently, Greedy starts by matching the I — 1 common substrings of 
A' and B' of length 1 . After this phase, each letter occurs in the remaining string 
exactly once, and thus the remaining matching is unique. The remaining letters 
tti in A' with the exception of ap are strings of length 1, and they give another 
I — 1 strings in the final partition. Now it is sufficient to bound the number of 
breaks forced in the string B” . There is a break before and after each letter bi, 
i = 1 (mod / -k 1 ), 1 < z < /^, as these are strings of length 1 in the remainder 
of B\ this gives 21 — 0(1) breaks. There is also a break before and after each 
letter 6 j, z = 1 (mod / — 1) as for such z, bibi+i is a consecutive pair in B” 
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but the possibly matching pair OiOi+i is not consecutive in A"; similarly ai-iai 
is consecutive in A!' but bi-ibi is not consecutive in B" . This gives 21 — 0(1) 
breaks. Finally, since I — 1 and I + 1 are relatively prime, only 0(1) breaks 
may be counted twice, by the Chinese remainder theorem. Altogether, Greedy 
partitions A = A' B” into 21 — 2 blocks of A' and — 0(1) blocks of B” , for the 
total of a>l — 0(1), and the lower bound of 3 on the approximation ratio follows 
by taking I arbitrarily large. 

Final Comments. We have established that Greedy’s approximation ratio 
is 0(n° ®®), but not better than I7(n°’^^). It would be interesting to determine 
the exact approximation of this algorithm. In particular, is it below, above, or 
equal to 0(^/n)7 Also, we have observed a difference between the performance 
of Greedy on 2-MCSP instances and 4-MCSP instances. Whereas the approx- 
imation ratio for 2-MCSP is 3, for 4-MCSP it is not better than l7(logn). The 
reason for this is, roughly, that for 2-MCSP every new cut (i.e., induced cut) is 
adjacent to a unique letter and, since Greedy does not make mistakes on unique 
letters, these new cuts do not induce any further cuts. However, for k > 2, new 
cuts may induce yet more new cuts again. An intriguing question is whether for 
4-MCSP the upper bound matches the l7(logn) lower bound, or whether it is 
higher? The question about exact approximation ratio of Greedy for fc-MCSP 
remains open even for fc = 3. 
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Abstract. We consider the problem of fitting metric data on n points 
to a path (line) metric. Our objective is to minimize the total additive 
distortion of this mapping. The total additive distortion is the sum of 
errors in all pairwise distances in the input data. This problem has been 
shown to be NP-hard by [13]. We give an O(logn) approximation for this 
problem by using Garg et al.’s [10] algorithm for the multi-cut problem 
as a subroutine. Our algorithm also gives an O(log^'^^n) approximation 
for the Lp norm of the additive distortion. 



1 Introduction 

One of the most common methods for clustering numerical data is to fit the 
data to tree metrics. A tree metric is defined on vertices of a weighted tree. 
The distance between two vertices is the sum of the weights of edges on the path 
between them. Here the main problem is to find a tree metric that represents the 
input numerical data as closely as possible. This problem, known as numerical 
taxonomy, has applications in various fields of science, such as linguistics and 
evolutionary biology. For example, in evolutionary biology tree metrics represent 
the branching process of evolution that leads to the observed distribution of data. 
Naturally, this problem has received a great deal of attention, (e.g. see [3, 14]). 

The problem of fitting data to tree metrics is usually cast as the problem of 
minimization of Lp{D,T)\ the Lp norm of additive distortion of the output tree 
T with respect to input data D. The input data is specified as an n x n matrix, 
where the entry denotes the distance between points i and j. Let Ty denote 
the distance between i and j in the output tree metric T. Then the Lp norm 
of additive distortion is Lp{D,T) = (Ei j ~ Ty^)^/P. Such a formulation 
was first proposed by [5] in 1967. In 1977, Waterman et al. [15] showed that if 
there is a tree metric T coinciding exactly with the input data D, then it can be 
constructed in linear time. In the case when there is no tree that fits the data 
perfectly, Agarwala et al. [1] used the framework of approximation algorithms 
to give heuristics with provable guarantees for the problem. They gave a 3- 
approximation to the Lqo norm of the additive distortion for fitting the data to 
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a tree metric. They reduced the problem to that of fitting the data to ultrametric, 
where each leaf is at the same distance from a common root. For ultrametrics, 
they used an exact polynomial-time algorithm for the Loo norm due to Farach et 
al. [8]. Agarwala et al. [1] showed that if there is a p-approximation algorithm 
for ultrametrics under the Lp norm, then it implies a 3p-approximation for tree 
metrics under the Lp norm. Our work is motivated by this problem. 

For a special case of fitting the data to a tree metric under Li norm, we 
make a simple observation. Suppose we know the structure of the tree along 
with a mapping of the points to the vertices of the tree. Then we can find the 
edge weights that minimize the Li norm of additive distortion using linear pro- 
gramming. However, finding the topology of the tree is the hard part. Therefore, 
we consider a special case of the problem in which we restrict the structure of 
the output tree to be a path. For this special case, we give an approximation 
algorithm. We believe that our techniques can be extended to solve the original 
problem. The main result of this paper is the following theorem. 

Theorem 1. There is an O {\og^ ^ ^ n)- approximation algorithm for the problem 
of fitting metric data to a line metric to minimize the Lp norm of additive 
distortion for p > 1. 

1.1 Related Work 

Note that fitting points to a path metric is equivalent to fitting points on a 
real line, where the distances in the real line are defined in a natural way. The 
special case of the problem for the L^o norm (i.e. with p = oo) was considered 
by Hastad et al. [11]. They gave a 2-approximation for it. 

For fitting points to a line, a well-known result due to Menger (see e.g. [6]) 
gives the following four point criterion. The four point criterion says that, if 
every subset of size 4 can be mapped into the real line exactly, then all the 
points can be mapped into the line exactly. An approximate version of Monger’s 
result was given by Badoiu et al. [2]. They proved that if every subset of size 
4 can be embedded into the line with the Loo norm of the additive distortion 
being at most e then all the points can be embedded with the Loo norm of the 
additive distortion being at most 6e. 

In a related work, Dhamdhere et al. [7] and Badoiu et al. [2] independently 
studied average distortion of embedding a metric into path metrics. The objective 
was to minimize subject to > Dy for all i,j, where L>y is the 

distance between points i and j in the input and is the distance between 
them in the output path metric. They gave a 14-approximation for the problem. 
While their problem has additional constraint fij > Dy , their objective function 
is easier than the L\ norm of the additive distortion. We do not know how to 
minimize the Lp (or even the Li) norm of additive distortion under the additional 
constraint. However, for the special case oi p = oo, we would like to point out 
that the algorithm for min-excess path problem due to Blum et al. [4] gives a 
2 -|- e approximation. The min-excess path problem asks for a path from a source 
s to a destination t that visits at least k vertices and minimizes the objective 
l{path) — d{s,t), where l{path) is the length of the path. 
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1.2 Techniques and Roadmap 

In Section 2, we define our problem formally. In Section 3, we show how to reduce 
the problem to that of finding the best r-restricted mapping. An r-restricted 
mapping of the input points into a line is one in which the distances of all 
points in the line from point r are same as that in the input. We show that this 
problem is equivalent to a two-cost partition problem. In Section 4, we give an 
approximation for this problem via the multi-cut problem [10]. 

2 Problem Definition 

Consider a set of n points, denoted by [n] = {1, 2, . . . , n}. The input data consists 
of an n X n matrix Dnxn- The entry Dij denotes the distance between points 
i and j. We assume that all the entries of D are non-negative and that D is 
symmetric^. Furthermore, we assume that Du = 0 for all i. 

Let / : [n] ^ IR denote a mapping of the input points to the real line. 
Distance between images of points i and j in the line is given by fij = ]/(*) — 
/(j)|. The total additive distortion (in the Li norm) is given by 

L,{D,f) = J2\D^J-fij\■ 

Generalizing this, we can write the Lp norm of the additive distortion as 

Lp(D,/) = (^|Ai-/yr)c 

'id 

The goal is to find a map / that minimizes the Li{D^ f) (or more generally 
Lp{DJ)). 



3 Approximation for Lp Norm 

In this section, we give an approximation algorithm for minimizing the Lp norm 
of the additive distortion. 

In Lemma 1 , we will show that it is sufficient to look at r-restricted mapping 
of the points into the real line. The problem of finding an optimal r-restricted 
mapping can be cast as a kind of partition problem given the characteristics of 
the real line. 

3.1 r-Restricted Mappings 

Let r be a point in the input. A mapping / of the input points to the real line 
IR is an r-restricted mapping, if distance on the line of all points from r is same 
as that in the input. Formally, D^i = |/(r) — /(*)] for all i. 

^ Our results hold even if the input distances in D„xn do not satisfy triangle inequality, 
i.e. even if D is not a “metric”. 
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We will denote an r-restricted mapping by We next show that there is 
always a “good” r-restricted mapping. This will enable us to focus only on r- 
restricted mappings which are easier to handle. Agarwala et al. [1] prove a similar 
lemma for tree metrics. We adapt their proof for the case of line metrics. 

Lemma 1. There exists a point r among the input points such that there is an 
r-restricted mapping /” that is within a factor of 3 of the optimal mapping for 
the Lp norm of additive distortion, for all p> 1. 

Proof. Let f* denote an optimal mapping of the input points to the line for the 
Lp norm of additive distortion. We will modify the optimal solution to produce 
a mapping /* for each point i in the input. To produce the restricted mapping 
/*, perturb the distances in f* so that it becomes i-restricted. In particular, if 
f*{j) < f*{i) for some j, then set f {j) = f*{i) - Aj and if f*{j) > f*{i), set 
/*0) = /*(*) + Dij. Our mapping /* maps point i to f*{i). It maps rest of the 
points according to their distance from i, while maintaining their order to the 
left or right of point i in the optimal mapping f* . 

Let 6jk denote \Djk~fjk\- We can write the additive distortion of the optimal 
mapping as Lp{D, /*) = ^ From the construction of the map /*, it 

follows that |/*j, - fjf.\ < iij + €ik- 

Now we bound the additive distortion of /* in terms of Cjk’s. For all j, k we 
have, 

\D,k-fU<\fIk-fik\ + \Djk-f*k\ 

— fo ^ik) + ijk (1) 

Note that \xf' is a convex function of a: for p > 1. Therefore, Equation (1) 
gives us the following: 

\Djk — fjk\^ < (a + Uk + ^jkY 

+ ( 2 ) 

By an averaging argument, we can say that 

i n 

We use Equation (2) to bound the sum 

n n 

vn./r < E E + 4 + 4) 

i—1 2=1 j,k 

S3'n-E4 

j.k 

= 3Pn-Lp{D,rr 

Therefore, min^ Lp{D, /*) < 3 • Lp{D, /*) which proves the result. 
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3.2 Algorithm 

The result of Lemma 1 proves that it is sufficient to consider r-restricted map- 
pings (with a loss of 3 in the approximation factor). Next we describe the algo- 
rithm that implements this idea. 

Algorithm A: 

1. For each point r = l,2,...,n, find (approximately) the best r-restricted 
mapping /’'. 

2. Output a mapping that has the smallest additive distortion among these 
mappings. 

By Lemma 1, the additive distortion of the output of Algorithm A is within 
a factor of 3 of the optimal additive distortion. As we point out in Section 5, 
finding best r-restricted mapping is NP-hard. Therefore, we approximate the 
optimal a-restricted mapping within a factor of O(log^^^n). From the following 
observation it follows that the overall approximation factor of our algorithm will 
be 0(log^^^ n). 

Lemma 2. If p is the approximation factor of the algorithm for r-restricted 
mapping, then the solution produced by Algorithm A will be a 3p approximation 
for the additive distortion. 

4 Approximating r-Restricted Mappings 

Let / be an r-restricted mapping. Without loss of generality, we can assume 
that /(r) = 0. Let V\ = {i \ f{i) < 0} and V 2 = {i \ f{i) > 0}. Note that 
[n] = V\U {r} U ¥ 2 - Note that, the mapping / is fully characterized by the 
partition V\ U V 2 of [n] — {r}. Hence, the problem of finding the best r-restricted 
mapping is equivalent to the problem of finding the partition ofV = [n] — {r} that 
has minimum additive distortion. Henceforth, we will think of the problem as 
that of partitioning the input set of points to minimize the cost of the partition, 
i.e. the additive distortion. For simplicity, we describe the argument for p = 1. 
The other cases (p > 1) are similar. 

Consider a partition Vi U V 2 induced by an r-restricted mapping /. We can 
write an expression for its cost as follows. Consider two points x and y. If they 
both belong to the same side of the partition, then the contribution of the pair 
{x,y} to the cost of the partition is c{x,y) = \Dxy-fxy \ = D^y -\f(x) - f{y) \ = 
\Dxy — \ Drx — DryW- On the other hand, if x and y belong to different sides of the 
partition, then the contribution is c'(x, y) = \Dxy — fxy \ = \Dxy — \f{x) — f{y)\\ = 
\Drx + Dry — Dxy\- Notc that c(x, y) and c'(x,y) are completely determined from 
the input matrix D„xn- 

Therefore, we can think of the problem as a graph partitioning problem where 
each edge has two costs c(-) and c'(-) associated with it. The objective function 
is 

c(x,y)-k c{x,y) 

x,y on same side x,y on different sides 
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4.1 Two-Cost Partition Problem 

We are given a complete graph G = {V, E) with two cost functions c and c' . 
We want to find a partition of the vertex set V = Vi U V2 which minimizes 

E*=i,2 ^'(m, v). 

Note that, if c(u,v) = 0 for all u,v, then the problem reduces to finding a 
minimum cut in the graph. On the other hand, if c'{u, v) = 0, then the problem 
is the well known edge deletion for graph bipartition problem (BIP) [12]. Our 
algorithm generalizes the algorithm for graph bipartition given by [12, 10]. The 
basic idea is to create two copies of each vertex to go on different sides of the 
partition. To ensure that they are on different sides, we designate each pair as a 
source-sink pair in the multi-cut subroutine. 

Algorithm B: 

1. Create an auxiliary graph G' from the graph G as follows. 

(a) For each vertex u in the graph G, G' has two vertices: u and u' . 

(b) For each edge (u,v) we create 4 edges in G': {u,v),{u,v'),{u' ,v) and 
{u',v'). 

(c) The edges in G' have weights, denoted by l{-, •). Set l{u,v) = l{u',v') = 
c(u,v) and l{u,v') = l{u',v) = c'{u,v). 

2. Use an approximation algorithm for the multi-cut problem (E.g., [10]) as 
a subroutine to find a multi-cut in graph G' with (u,u'), for all u, as the 
source-sink pairs. Let S be the set of edges in the multi-cut returned by the 
subroutine. 

3. Construct a set of edges T as follows. If {u, v} or {u' , v'} is chosen in S, then 
include both in T. Similarly, if {u, v'} or {«', w} is chosen, then include both 
in T. 

4. Find a bipartition V[ U V2 of vertices of G' so that T contains all the edges 
going across the partition^. 

5. Output the partition Vi U V 2 , where Vi = V( C\ V. 

The intuition behind this algorithm is as follows. For the cut represented by 
T, we will show that we can get a partition of vertices in graph G' such that only 
one of u and u' is in one partition. From the partition of G', we get a bipartition 
of G. The cost of the bipartition of G is related to the cost of multi-cut obtained 
by above algorithm in the graph G'. We prove this in the next lemma. 

Lemma 3. Algorithm B returns a partition V = V{ UU 2 of graph G' , such that 
if u G V{, then v! G Vf and vice versa. Moreover, yeV most 

twice that of the multi-cut found after step 2 by Algorithm B separating each u 
from u' . 

Proof. Consider the set S of edges found by the multi-cut subroutine whose 
removal separates each u from u'. For each edge (x, y) G S, we also include its 
“mirror” edge in T. i.e. if (x,y) G S, then {x' ,y') G T from the graph. Note 

We will show how to do this in the proof of Proposition 1. 



2 
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that, the cost of an edge and its “mirror” edge is same (i.e., l{x,y) = l{x',y')). 
Therefore, the cost of the edges in T is at most twice the cost of edges in S. 

Now we show that removal of the edges in T breaks the graph in two parts 
with the desired property. Consider the graph G'\T. Construct a graph H whose 
vertices represent the connected components in G' after removing the edges in 
T. Two vertices hi and /12 in H are connected to each other if the corresponding 
connected components in G' have vertices x and x' . 

In Proposition 1, we prove that the graph H is bipartite. Now we can use 
graph H to construct a partition V = V( U V 2 in graph G' . Since the vertices 
in graph H were connected components in graph G' , there are no edges crossing 
the partition V{ U V 2 in graph G' . Moreover, bipartiteness of graph H means 
that each pair of vertices x and x' in graph G is split in the partition. The cost 
of this partition is at most 2 times the cost of the multi-cut. 

Proposition 1. The graph H defined in the proof of Lemma 3 is bipartite. 

Proof. For the sake of contradiction, assume that H has a cycle of odd length. 
Consider three consecutive vertices rt, v and w in this odd cycle. Let v be con- 
nected to u and w. 

Let X be a vertex of G' that belongs to the connected component u and defines 
the edge {u,u} in graph H. Therefore, x' is the component v. Similarly, let y be 
a vertex in component w and y' be the corresponding vertex in component v. 
Since x' and y' are in the same connected component v, there is a path x' y' 
that lies completely inside the component v. Since we didn’t remove any of the 
edges on the path x' y' , all the mirror edges haven’t been removed either. 

Therefore the the mirror path x y connects x and y. This contradicts the fact 
that x and y were in different connected components. 

This proves that the graph iL is a bipartite graph. 

Lemma 4. The cost of the optimal multi-cut is a lower bound on the cost of 
partition of graph G. 

Proof. Consider a partition V = Vi U V 2 of graph G. From this, we can construct 
a partition of the vertex set of G' . Let V[ = Vi U {x' | x G V 2 } and Vf = V'\V[. 
Then, removing all the edges in G' crossing this partition ensures that no vertex x 
is connected to its counterpart x'. i.e. The set of edges going across the partition 
is a multi-cut. The cost of these edges is exactly the cost of the partition of G. 

Recall that GVY algorithm for multi-cut [10] is an O(logfc) approximation 
for k terminals. Here we have n terminals. Therefore by Lemmas 3 and 4, we 
get an 0(log n) approximation for the best r-restricted mapping. Along with 
Observation 2 give us an O(logn) approximation for the Li norm of additive 
distortion. 

To get an approximation algorithm for the Lp norm, we modify the costs in 
the two-cost partition problem as follows. Let c(x, y) = iD^^y — \Dax — Day\\^ and 
c'{x,y) = \Dax + Day ~ Dxy\^. With these costs. Algorithm B gives an O(logn) 
approximation for Lp{D, /“)^. Therefore, for the Lp norm of additive distortion, 
we get an 0(log^^^ n) algorithm. 
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5 Open Questions 

We can show that the problem of finding the best r-restricted mapping is NP- 
hard by reducing the edge deletion for graph bipartition (BIP) [9] to it. Consider 
a graph G. Let V (G) = n. We construct a distance matrix D on n + 1 points 
V{G) U {a}. Set the diagonal entries D^x to 0. Set Dax = 1/2 for all x S V{G). 
For all {x,y} G E{G), set Dxy = 1. Set the rest of the entries to 1/2. Consider 
an r-restricted mapping. Let V (G) = Vi U V 2 be the partition induced by the 
r-restricted mapping. Then the cost of the r-restricted mapping is V 2 ) + 

(1/2)((2) — |if(G)|), where B(Vi,V 2 ) is the number of edges that need to be 
deleted to obtain Vi and V 2 as two sides of a bipartite graph. Therefore, finding 
the optimal r-restricted mapping corresponds to minimizing the number of edges 
deleted for making the graph G bipartite. This proves that finding the best r- 
restricted mapping is NP-hard. However, this reduction is not approximation 
preserving. So it does not preclude the possibility of a PTAS for this problem. 
Getting even a constant factor approximation would be quite interesting. 

In the proof of NP-hardness of r-restricted mapping problem, we used an 
input matrix D that does not satisfy the triangle inequality. For input matrix D 
that is a metric (i.e. it satisfies the triangle inequality), it might be possible to 
get a polynomial time algorithm for the best r-restricted mapping. 

Agarwala et al. [1] have shown that for the problem of fitting data to tree 
metrics, the best r-restricted tree metric is within a factor of 3 of the optimal 
solution. However, the problem of approximating the additive distortion of the 
best r-restricted tree is still open. 
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Abstract. We prove that there exists a universal constant c > 0 such 
that the Radio Broadcast problem admits no additive c ■ log^ n-approxi- 
mation, unless NP C BPTIME(rf^^°^^°^"^). For graphs of at most 
logarithmic radius, an 0(log^ n) additive approximation algorithm is 
known, hence our lower bound is tight. To the best of our knowledge, 
this is the first tight additive polylogarithmic approximation result. 



1 Introduction 

1.1 The Radio Broadcast Problem 

Definition and Motivation. Consider a synchronous network of processors 
that communicate by transmitting messages to their neighbors, where a proces- 
sor receives a message in a given step if and only if precisely one of its neighbors 
transmit. The instance of the Radio Broadcast problem, called radio network, is 
a pair (G = (V, E), s), s G V, where G is an unweighted undirected graph, and s 
is a vertex, called source. The objective is to deliver one single message that the 
source s generates to all the vertices of the graph G using the smallest possible 
number of communication rounds. The prescription that tells each vertex when 
it should broadcast is called schedule] the length of the schedule is the number 
of rounds it uses, and it is called admissible if it informs all the vertices of the 
graph (see Section 2 for a formal definition). From practical perspective, the 
interest to radio networks is usually motivated by their military significance, as 
well as by the growing importance of cellular and wireless communication (see, 
e.g., [KM-98,GM95,BGI91]). The Radio broadcast is perhaps the most impor- 
tant communication primitive in radio networks, and it is intensively studied 
starting from mid-eighties [GR-03,KP02,KP-03,GGR-00,GG-b02,GMS-03,I-02] 
[GM95,KM-98,ABLP91,BGI91,GW-87,GK-85]. 

* Part of this work was done while the first author was in School of Mathematics, 
Institute for Advanced Study, Princeton, NJ, USA, 08540. 
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From theoretical perspective, the study of the Radio Broadcast problem pro- 
vided researchers with a particularly convenient playground for the study of 
such broad and fundamental complexity-theoretic issues as the power and lim- 
itations of randomization, and of different models of distributed computation 
[BGI91,KM-98,KP-03]. In this paper we study the additive approximation thresh- 
old of the Radio Broadcast problem; we believe that our results show that this 
problem is of a particular interest from the stand-point of the theory of Hardness 
of Approximation as well. 



Previous Results. The first known algorithm for the Radio Broadcast problem 
was devised by Chlamtac and Weinstein in 1987 [CW-87]. That algorithm, given 
an instance (G, s) of the problem, constructs an admissible broadcast schedule 
of length 0{Rad{G, s) ■ log^ n) for this instance, where Rad{G, s) stands for the 
radius of the instance (G, s), that is, the maximum distance dais, v) in the graph 
G between the source s and some vertex v € V. Their algorithm is centralized, 
i.e., it accepts the entire graph as input. 

Soon afterwards Bar-Yehuda et al. [BGI91] devised a distributed random- 
ized algorithm that provides admissible schedules of length 0{Rad{G, s) ■ log n-l- 
log^n). Alon et al. [ABLP91] have shown that the additive term of log^ n in 
the result of [BGI91] is inevitable, and devised a construction of infinitely many 
instances (G, s) of constant radius that satisfy that any broadcast schedule for 
them requires G(log^ n) rounds. Kushilevitz and Mansour [KM-98] have shown 
that for distributed algorithms, the multiplicative logarithmic term in the re- 
sult of [BGI91] is inevitable as well, and proved that for any distributed algo- 
rithm for the Radio Broadcast problem there exist (infinitely many) instances 
(G, s) on which the algorithm constructs a schedule of length Q{Rad{G,s) ■ 
\og{n/ Rad{G, s))). Finally, the gap between the logn and \og{n/ Rad{G, s)) was 
recently closed by Kowalski and Pelc [KP-03] , and Gzumaj and Rytter [GR-03] . 

Gaber and Mansour [GM95] have shown that centralized algorithms are ca- 
pable of constructing much shorter schedules, and devised an algorithm that 
constructs admissible schedules of length 0{Rad{G, s) -h log® n). Their result is 
also the current state-of-the-art in terms of the existential upper bounds on the 
length of broadcast schedules for instances of large radius. 

Remark: In [EK-03] we have improved the result of [GM95] by providing sched- 
ules of length Rad{G, s) -h 0{^ Rad{G, s) • log^ n) = 0{Rad{G, s) -I- log^ n). 

Since, obviously, any schedule for an instance (G, s) requires at least 
Rad{G,s) rounds, the algorithms for the Radio Broadcast problem 
[GW-87,BGI91,GM95, KP-03, GR-03] can be interpreted as approximation algo- 
rithms for the problem. The state-of-the-art in this context are the results by 
Bar-Yehuda et al. [BGI91] for instances of small radius, and the result of Gaber 
and Mansour [GM95] for instances of large radius. (We term such an approxi- 
mation guarantee “a semi-additive” , as opposed to “a purely additive” guarantee 
for which the multiplicative factor of the optimum should be equal to 1.) 

In terms of the lower bounds, the NP-hardness of the Radio Broadcast prob- 
lem was shown by Ghlamtac and Kutten [GK-85] already in 1985. The authors 
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of the current paper have shown [EK-02] that there exists a constant c > 0 
such that the Radio Broadcast problem cannot be approximated with a (purely) 
additive approximation ratio of clogn unless NP C 



1.2 Our Results 

In this paper we show that there exists a constant c > 0 such that the Radio 
Broadcast problem cannot be approximated with a (purely) additive approxi- 
mation ratio of clog^n, unless NP C This result holds 

even for the Radio Broadcast problem restricted to graphs of constant radius. We 
believe that our reduction contains lower bound ideas for radio model context 
that should have further applications. 

Observe also that our results together with [BGI91] fully determine the ad- 
ditive approximation threshold (up to constants and with randomization al- 
lowed) of the Radio Broadcast problem restricted to instances (G, s) of radius 
Rad{G,s) = O(logn). 

Hence we have demonstrated that the approximation behavior of the Ra- 
dio Broadcast problem is best-described in terms of an additive polylogarith- 
mic threshold. It appears that there is no other problem whose approximation 
threshold is known to exhibit a similar behavior. We believe that demostrating 
the existence of a new pattern of approximation behavior enriches the theory of 
Hardness of Approximation, and constitutes the main contribution of this paper. 



1.3 Proof Techniques 

The proof of our lower bound incorporates the combinatorial construction of 
Alon et al. [ABLP91] in the reduction of Lund and Yannakakis [LY94]. The 
latter reduction reduces the Label-Cover problem to the Set Cover problem, and 
establishes the logarithmic hardness of approximation of the latter. However, 
the importance of this reduction extends far beyond this specific proof, and 
the reduction was used as a building block for many inapproximability results 
[ABSS93,EK-02,DCKR03,HK-03] . 

However, it seems that the reduction of [LY94] alone cannot provide greater 
than logarithmic for the Radio Broadcast problem. The second crucial building 
block of our lower bound is the result of Alon et al. [ABLP91]. Their construction 
provides a bipartite graph in which it requires f?(log^ n) rounds to deliver the 
message from the left-hand side of the graph to its right-hand side. 

To explain how we incorporate the construction of [ABLP91] into the re- 
duction of [LY94], we first outline the structure of the reduction of [LY94]. For 
each edge e of the input instance of the Label-Cover problem, the reduction of 
[LY94] constructs a bipartite graph (5g,Mg), where each vertex a; G is con- 
nected to one half of the vertices of the set Mg. Furthermore, these connections 
are ingeniously arranged in such a way that the following condition is satisfied: 
for any k = o(logn) vertices xi,X 2 , ■ ■ ■ ,Xk G 5g, and for any choice of sets 
Xi,X 2 , . . . , Xk such every Xi is either equal to the set Ni of the vertex Xi in Mg 
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or to its compliment Mg\Ni, the set Uf=i does not cover the vertex set Mg. 
Such a bipartite graph is called a set system. 

The reduction of [LY94] also provides each such set system (Sg,Mg) with a 
simple “trapdoor” that enables to cover the set Mg with only two sets, specifically, 
the set Ni and the set Mg\Ni. These trapdoors are geared to enable to construct 
small solutions for the instances of the Set Cover problem whenever the original 
instance of the Label-Cover problem is a YES-instance, but, however, cannot be 
used if the original instance is a NO-instance. 

The basic idea of our proof of the polylogarithmic lower bound for the Ra- 
dio Broadcast problem is to replace the set systems of [LY94] by the bipartite 
graphs of [ABLP91]. To accomplish it, one, however, needs to devise analogous 
trapdoor to the construction of [ABLP91], and doing it without ruining the 
(rather delicate) properties of [ABLP91]. It is, therefore, not surprising that our 
construction is quite complex, and its analysis is technically very involved. 

Comparison with [EK-02]: The reduction of [EK-02] can be seen as a (non- 
trivial) adaptation of the reduction of Lund and Yannakakis to the Radio Broad- 
cast problem. The reduction in the current paper is far more complex both con- 
ceptually and technically, particularly because it incorporates the construction 
of [ABLP91] in the reduction of Lund and Yannakakis. 

2 Preliminaries 

2.1 The Radio Broadcast Problem 

We start with introducing some definitions and notations. 

Definition 1. The set of neighbors of a vertex v in an unweighted undirected 
graph G{V,E), denoted Tg{v), is the set {u & V \ (v,u) G B}. For a subset 
X C V, the set of neighbors of the vertex v in the subset X, denoted Fg(v,X) 
(or r{v,X) when the graph G is clear from the context), is the set {u G X \ 
\v,u) G E}. 

Definition 2. Let G = (V, E) be an unweighted undirected graph, and R C V 
be a subset of vertices. The set of vertices informed by R, denoted I{R), is 
I{R) = {t’ I 3!x G R s.t. v G /gI^e)} (the notation 3\x stands for “there exists 
a unique x”). For a singleton set R = {x}, I{R) = d({cc}) = I{x) = Fg{x). 

A sequence of vertex sets IT = (i?i, i? 2 , ■ • ■ , Rq), q = 1, 2, . . ., is called a radio 
broadcast schedule (henceforth referred as a schedule) if Ri+i C ljj=i ^i^j) for 
every z=l,2, ...,g— 1. The set of vertices informed by a schedule II, denoted 

/(7T),is J(7T)=Ufle7i^(^)- 

An instance of the Radio broadcast problem ^ is a pair (G = (V,E),s), 
where G is a graph, and s G P is a vertex. The goal is to compute an admissible 
schedule IT of minimal length. The value of an instance Q of the radio broadcast 
problem is the length of the shortest admissible schedule IT for this instance. 
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2.2 The MIN-REP Problem 

Definition 3. The MIN-REP problem is defined as follows. The input consists 
of a bipartite graph G = (Vi, V 2 ,E). In addition, for j = 1,2, the input contains a 
partition Vj of Vj into a disjoint union of subsets, Vi = UagVi ^2 — Usep2 
The triple Ai = {G,V\,V 2 ) is an instance of the MIN-REP problem. The size 
of the instance is n = |Pi| + IV 2 I. An instance G as above induces a bipartite 
super-graph G = (Vi,V 2 ,E) in which the sets A and B of the partition serve 
as the vertices of the super-graph. The edges of the super-graph are E(A4) = 
E = {{A,B) G Vi X V 2 \ a G A,b G B,{a,b) G E}. In other words, there is a 
(super-) edge between a pair of sets A G V\, B G V 2 if and only if the graph G 
contains an edge between a pair of vertices a G A, b G B. 

Denote V = Vi U p 2 - A pair of vertices X\,X 2 G Vi U V 2 is called a match- 
ing pair with respect to a super-edge e = {A, B) G E (henceforth, e-m.p. j if 
(xi,X 2 ) G E and either xi G A and X 2 G B or vice versa. 

A subset C C Vi U V 2 of vertices is said to cover a super-edge e = {A, B) if 
it contains an Gm.p.. A subset C CV 1 UV 2 that satisfies \G D X\ = 1 for every 
X G V is called a MAX-cover. In other words, a MAX-cover G contains exactly 
one vertex from each super-vertex. An instance Ai of the MIN-REP problem is 
called a YES-instance if there exists a MAX-cover that covers all the superedges. 
Such a MIN-REP solution is called a perfect MAX-cover. For a positive real 
number t > 1, an instance Ai of the MIN-REP problem is called a t-NO-instance 
if any MIN-cover G of the instance Ai covers less than \E\/t super-edges of Ai. 

We also impose several additional (somewhat less standard) restrictions on 
the set of instances of the MIN-REP problem. 

1. All the super-vertices X gV are of size |A| = n^-'^ j2. 

2. The set of super-edges has sufficiently large cardinality, specifically, \E\ = 

® • log^n). (Note that by restriction 1, just for the super-graph to be 
connected, \E\ has to be at least 2n° ® — 1.) 

3. The Star property: For every super-edge e = {A,B) G E, A GVx and 
B C V 2 and every vertex b G B there exists exactly one vertex a G A, 
denoted e(6), such that (a, 6) G E. This property is called the star property. 

Theorem 1. [R98] No deterministic polynomial time algorithm may distinguish 
between the YES-instances and the log^° n-NO -instances of the MIN-REP prob- 
lem, unless NP C DTIME(nP^^°^^°^'^">), even when the instances of the MIN- 
REP problem satisfy the conditions (l)-(3). 

3 Reduction 

We next describe our randomized reduction from the MIN-REP problem to 
the Radio broadcast problem. This reduction always maps a YES-instance of 
the MIN-REP problem to an instance of the Radio broadcast problem of value 
T -\- O(logn), and with high probability over its own coin tosses the reduction 
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maps a NO-instance of the MIN-REP problem to an instance of the Radio broad- 
cast problem of value T -|- l7(log^ n), where T = 1,2,... is a parameter of the 
reduction. 

Consider an instance M = (G, ^ 1 ,^ 2 ), G = (Vi,V 2 ,E) of the MIN-REP 
problem with Vi = UyieVi ^2 = UsePs reduction constructs an in- 

stance Q = G(A4) = (G, s), G = {V, E), s e P, of the Radio broadcast problem 
in the following way. 

Let N = The vertex set V of the graph consists of the source s, and 
the disjoint vertex sets Vi and V 2 (i.e., V = {s} U Vi U V 2 with s^Vi U V 2 , and 
Vi n V 2 = 0). It is convenient to visualize the source s on the ’’left-hand” side 
of the graph, the vertex set Pi in the ’’middle”, and the vertex set P 2 on the 
’’right-hand” side of the graph. The vertex set Vi contains N copies of every 
vertex a; G P = Pi U P 2 ; these copies are denoted cpi{x),cp 2 {x),. . . ,cpj^{x). 
The set of all copies of a vertex x, {cpi(x), cp 2 (x ), . . . , cp^(x)}, is denoted by 
Xx- For a subset X CV, let xx denote xjc = Xx- Let J denote the set of 

indices {0.4 log n, 0.4 log n -|- 1, . . . , 0.6 log n}. For an index j G J, and a vertex 
a; G P, let Xx{j) denote the subset of the set Xx that contains only the first 
J = 2^ copies of the vertex x. I.e., Xx{j) = {cPi(x), cp 2 (x), . . . , cpj{x)}. These 
copies are also called the j -relevant copies of the vertex x. For a subset X CV, 
let xx{j) = [JxexXxij)- 

The vertex set V 2 is of the form V 2 = Usei: where the ground sets Mg 

are disjoint, and all have equal size. Each ground set Mg is a disjoint union of 
the sets Mg{j,q), j G J, q G [L], with L = 2 ■ n'”°, and cq is an integer positive 
universal constant that will be determined later. The sets Mg{j, q) are all of 
equal size M = n‘^°, for the same constant Cq. 

The edge set E of the graph G contains edges that connect the source s to the 
vertices of the set Vi , and edges between the vertices of Vi and V 2 . Specifically, 
the edge set E is formed in the following way. 

1 . Connect the source s to all the vertices of the set Vi . 

2. For every super-edge e = {A, B) G E, for every pair of indices j G J, q G [L] 

do 

(a) For every vertex a G A 

i. Let Hg^a{j,<l) be an exact random half of the set Mg(j,q). 

In other words, every subset H C Mg{j, q) that has cardinality 
\Mg{j,q)\/2 (for simplicity we ignore all the Integrality issues) has 
the same probability to be chosen to serve as Hg^a{j, q)- 

ii. Let q) be an exact random half of the set Elg^a{j, q)- 

Set Hfl U, q) = Hg,a (j, q)\Hi]l (j, q) . 

(b) i. For every vertex bG B, set q) = Mg{j, g)\iLe,g(h)(j, q). 

ii. Let E[A)e b(j, q) be an exact random half of the set Efg b(j, q)- 

SetiLg(j{g)=if,,,(j{g)\ijg(j,g). 

(The steps 2a-2b will be referred as the exact partition step.) 
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(c) For every vertex x G A\J B let x^x\j) be a subset of Xx{j) that is 
formed by picking each element of the set Xx{j) with probability 1/2 
independently at random. 

Set Xx\j) = Xx{j)\Xx\j)- 

(The step 2c will be referred as the binomial partition step. 

(d) For every vertex x & AU B, and index i G {1, 2} do 



(e) 



Let Xx\j) = {cp'^-^\x,j),cp)^\x,j),...,cp^l\x,j)), with y = x{x,i,j) 

= \Xx\j)\. 

Let S'y be the set of all permutations of the set [y] . 

Choose a permutation a Gr uniformly at random from the set S'y. 
Partition the set q) arbitrarily into y disjoint subsets q, £), 

^G[X]- 

For every index £ G [y], connect the vertex cp^^\x,j) to every vertex v 
in the set P^"l{j,q,(r{£)). 

(The step 2d will be referred as the random permutation step.) 

i. For every copy ep{x) G Xx\j), and every vertex v G H^'^^{j,q), with 
probability 1/J insert the edge {x,v) into the edge set E, indepen- 
dently at random. 

ii. Do the same for every copy cp{x) G Xx\j), and every vertex v G 

Me{j,q)\He,xi.j, q)- 

iii. Do the same for every copy ep{x) G Xx\Xx{j), and every vertex 
V G Me{j,q). 

(The step 2e will be referred as the mixing step.) 



Ji){ 



Ai) 



(i){ 



We need some additional notation. For a pair of vertices a; G Vi and v G 
V 2 , let {x C v) denote the event {{x,v) G E), and let {x Af v) denote the 
complimentary event {{x,v)^E). For a subset X CV\ and a vertex v G V 2 , let 
{X AC v) denote the event (Va: G X, {x, v) & E), and, analogously, let {X AN v) 
denote the event (Va; G X, (x,v)^E). For a pair of subsets X C Vi, U C V 2 , let 
{X AC U) denote the event (Va: G X,v G U, (x,v) G E), and, analogously, let 
(X AN U) denote the event (Vx G X,v G U, (x,v)^E). 

For a subset X C Vi, a vertex v G V 2 , and a positive integer £=1,2,..., let 
{X (> £C) v) denote the event (|{x G X | (x,u) G E}\ > £). Analogously, let 
(A (< iC) v) denote the event (|{x G A | (x, v) € E}\ < £), and let (A (iC) v) de- 
note the event (|{x G A | (x, x) G £’}| = £). We also use the notation (A XN T v) 
for the event (A (1C) v); in this case we say that the subset A informs the vertex 
V. For the complimentary event we use the notation (A NXNT v), and say that 
the subset A does not inform the vertex v. For a subset U C V 2 , we use the 
notation (A XN T U) for the event (Vx G U, (A XN T x)); in this case we say 
that the subset A informs the subset U. For the complimentary event we use the 
notation (A NXNT U), and say that the subset A does not inform the subset 
U. For a collection of subsets II = (Ai, A 2 , . . . , Xt) of Vi and a vertex x G V 2 
(respectively, a subset U C V 2 ), we say that the collection 77 informs the vertex 
V (resp., the subset U), and denote it by (77 XN T v) (resp., (77 XN T U)), if at 
least one subset X G II informs v (resp., U). For the complimentary event we 
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use the notation (77 NXMT v) (resp., (77 MXN T U) and say that the collection 
77 does not inform the vertex v (resp., the subset U). 

For a fixed super-edge e = {A, B) G E, and a vertex a; G ^ U 77, let star{x) 
denote the subset of vertices of A U 77 that belong to the same star as x with 
respect to the super-edge e. For a subset S' C ^ U 77, let cp{S) denote the set of 
all the copies of the vertices of S, i.e., cp{S) = {cp^(x) | x G S}. 

4 Analysis 

In this section we analyze the reduction described in the previous section. 

4.1 Basic Properties 

Definition 4. For a super-edge e = {A,B), a subset S C XA U xb is called 
(e, j)-partial if for every e-m.p. (a,b), the set S contains at most j-relevant 
copies of the vertex a or b. 

Consider some pair of vertices (a,b) with a G 4 and b G B, for some pair 
of super- vertices A G Vi and B G ¥ 2 - Let j G J, q G [L] be a pair of indices. 
Consider a subset S C Xa{j) U Xb{j), and a vertex v G Hg^b{j,q)- Recall that 
the set XbU) is split into the disjoint union of the subsets and X^b^j) by 

a binomial random partition (see Section 3, step 2c). The set Hb,b{j,q) is split 
into a disjoint union of the two subsets 77 ^^ (j^ g) and Flf'l (j, g), and these two 
subsets have the same size M/2. Let = 5 C x\^\j)j S 2 = S H x^\j)- 

Let Si = |5i|, S 2 = |52|, w = |x[^^(j)| and u = |5 C Xa(j)|- Note that si, 
S 2 , and w are random variables, and u is a fixed value that does not depend on 
random coins. 

Lemma 1. For a super-edge e = {A, 77) G 77, a pair of vertices a G A, b G B , 
a pair of indices j G J , q G \L], a subset S C Ya( 7 ) U Vh(7’) and a vertex v G 
M,{j,q),V{SANv) > (l-l/2^-)l5|.^^_^^H^.p(|5i| = si,|x«(j-)|^ 

Consider a subset S C xa(j) Uxb(j)- Let S = S{S) be the subset of vertices 
of 4 U 77 that have copies in the set S. Note that the set S decomposes into a 
disjoint collection of the subsets S{a) = cp{star{a)) C S, for different vertices 
a G S. For a yet more general subset S C XA U xb, we will use the notation 
Sj = Sj{S) for the subset of vertices of 4 U 77 that have j-relevant copies in the 
set S. 

We will use the notation x to denote an arbitrary vertex x G S, both for 
X G A and cc G 77. Let PN^iv) denote the subset of copies of the vertex x that 
have a non-zero probability of being connected to the vertex v G Me{j,q) by 
the random permutation step. In other words, let j{v) be the index j such that 
v G Mg{j,q). If u G Mg{j,q)\Hg^x{j,q), then set PN^{v) = 0. Otherwise, let 
iv(x) be the index i such that v G H^^^{j,q). Then set PN x{v) = 

Definition 5. Consider a fixed super-edge e = (A,B), a pair of indices j G J , 
q G [L], a subset S C xa U xb, and a vertex v G Mg{j, q). The exact partitions 
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of the set Me{j,q) into the unions of the subsets He,a{j,q) and He,b{j,q) for 
different e-m.p.s (a,b), the exact partitions of the sets He,x{j,q) into the unions 
of the subsets H~^l(j,q) and H~^l(j,q) for different vertices x & A\J B , and the 

binomial partitions of the sets Xx{j) into the unions of the subsets X^\j) and 
( 2 ') 

Xx \j) for different vertices x & A\J B are called safe with respect to the pair 
(5, v) if for every vertex x G Sj{S), |5 n PN x{v)\ < 2^~^. 

Lemma 2. Let S be an {e,j)-partial subset of XA U xb (see Definition 4) of 
size |5| < 2-^ lnn, and v G Me{j,q) be a vertex. The probability that the exact 
partitions of the set Mg(j,q), and the binomial partitions of the sets Xx{j) are 
safe with respect to the pair (S,v) is at least 1/n®. 

Corollary 1. With probability at least 1/n®, for every vertex x G S, \PN a:{v) n 
5 | < 2 ^- 3 . 

4.2 Large Rounds and Their Effect 

In this section we consider the probability space that is determined by the ran- 
dom permutation and mixing steps (steps 2d - 2e) of the reduction, and, in 
particular, the partitions of the sets Xx{j) are fixed for all the vertices x G AiJB 
(and the index j G J is fixed as well). 

Definition 6. We say that a round R is j-large with respect to a super-edge 
e = (A,B) if Rb = RC\ {xa U xb) is of size greater or equal to c • 2-’+^ Inn, 
where c = 2cq + 2 (cq is a universal constant mentioned in Section 3; it will 
be determined later). Otherwise, the round is called j-small with respect to the 
super-edge e. 

A schedule II is called short if its length is at most log^n/100. 

Consider a short schedule II. Let 7Tg(j) (respectively, IIg{j)) be the subsched- 
ule of n that contains only j-small (resp, j-large) rounds with respect to the 
super-edge e. Note that a vertex v G Ms(j) is informed by the schedule IT if 
and only if either the subschedule 7rg(j) or the subschedule n^{j) informs v (or 
if both do). 

Let Vs{TTs{j)) = (xaUxb) = and, analogously, 

^s{TIs{j)) = Ufi67Ie(i) 

Next, we show that the probability that the subschedule !!&{)) informs a 
significant number of vertices is negligibly small. 

For a fixed super-edge e = {A,B) G E, and a fixed index j G J, consider a 
subset M' C Mg{j) such that for every index q G [I], \M' n Mg{j,q)\ < 1. Such 
subsets are particularly convenient for the analysis, since for a pair of distinct 
indices qi,q 2 G [I], and a pair vertices vi G Me{j, qi), V 2 G Mg{j, 52 ), the events 
{x C ui) and (x C V 2 ) are independent. 

Lemma 3. For a fixed short schedule 77, with probability at least 1 — , 

there is no subset M' C Me{j) of size N = n that satisfies |Mg(j, q) n M'\ < 1 
for every index q G [I], such that the subschedule IIe{j) informs all the vertices 
of the subset M' . 
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4.3 Pivots 

We start with describing a combinatorial lemma from [ABLP91]. We will use 
this lemma in the rest of our analysis. 

Lemma 4. [ABLP91] Let G = (Vi,V 2 ,E) be a bipartite graph with \Vi\ = n. 
Let LI = {Ri, R 2 , ■ . ■ , Rt) be a collection of at most subsets of V\. Then 

there exists a subset S of the vertices on the left-hand side of the graph (S C V\), 
and an index j € J , such that 

1. |5| < 2 I ■ logn. 

2. Let LI' = {R'l^ i? 2 , ■ • •) = II\S. Then for every subset R' G LI' , |i?'| > 2^ . 

3. Let fk be the number of subsets R' with cardinality 2^+^ < |i?'| < 2^+^+^. 
ThenJ2k>o^ < logn. 

Fix a super-edge e = (A,B), and consider the subgraph of the graph G = 
(V,E) induced by the vertex set {xa U \b) U Mg. Let II\g be the schedule 
n restricted to the set xa ^ Xb- Apply Lemma 4 to this subgraph and the 
collection 7T|g of subsets of the set xa U xb, and let S and j be the subset 
and the index whose existence are guaranteed by the lemma. (Observe that 
IXA u xbI = ■\AUB\ = n.) 

Definition 7. (1) The index j as above is called the pivot of the super-edge e 
with respect to the schedule LI . (2) A round R is small with respect to the super- 
edge e if it is j-small with respect to super-edge e, and j is the pivot of e. (3) 
The schedule 77 is called e-proper if Vg{TTg{j)) is an {e, j) -partial set, where the 
index j € J is the pivot of the super- edge e with respect to 77. 

4.4 Small Rounds 

Consider a schedule 77 = {R\, R 2 , ■ ■ ■ ,Rt), t < (log^n)/100, and a super-edge 
e = {A,B). Let LI\g = {Ri\g, R 2 \g, . . . , Rt\g) , Rt\e = RiA (xa U xb), be the 
subschedule of 77 restricted to the super-edge e (for simplicity of notation we 
assume that for every index £ G [t], the set Ri\g is not empty; otherwise the 
number of rounds in the restricted schedule would be smaller than in the original 
one). 

Lemma 5. For a fixed super-edge e G E, and a fixed e-proper schedule LI, 
consider the L sets Mg{j,q), q G [L], For each index q G [L], let Xq denote the 
indicator random variable of the event (7Tg(j) TAfE Mg{j, q)). Let X = J^q^i Xq. 
Then V{X < 2n"° — n) > 1 — exp{—fl{n‘'°)). 

Lemma 6. For a fixed super-edge e, and a fixed short schedule LI, 

V{LI TMT Mg{j)) < exp{— L2{n"°)) , where j is the pivot of the super-edge e with 
respect to the schedule 77. 

Definition 8. A schedule IT is called proper if there exists a subset 77 C 77 that 
contains at least one half of all the super-edges, and such that the schedule IT is 
e-proper with respect to every super-edge e G 77. Otherwise, the schedule IT is 
called non-proper. 
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Consider a NO-instance. Let U denote the collection of all short proper 
schedules. 

Lemma 7. With probability at least 1 — exp{—f2{n^'^log^n)), there is no ad- 
missible schedule U G U . 



4.5 Deriving the Results 

In this section we glue together all the pieces of the analysis and derive our main 
results. The details are omitted from this extended abstract. 

Theorem 2. Unless NP C there exists a universal 

constant c such that no (deterministic or randomized) polynomial time algo- 
rithm may provide an additive (c-log^ n)- approximation guarantee for the Radio 
broadcast problem. 
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Abstract. For a prime p, max-21inp is the problem of satisfying as many 
equations as possible from a system of linear equations modulo p, where 
every equation contains two variables. Hastad shows that this problem 
is NP-hard to approximate within a ratio of 11/12 + e for p — 2, and 
Andersson, Engebretsen and Hastad show the same hardness of approx- 
imation ratio for p > 11, and somewhat weaker results (such as 69/70) 
for p — 3,5, 7. We prove that max-21inp is easiest to approximate when 
p — 2, implying for every prime p that max-21inp is NP-hard to approxi- 
mate within a ratio of 11/12 -|- e. For large p, we prove stronger hardness 
of approximation results. Namely, we show that there is some universal 
constant <5 > 0 such that it is NP-hard to approximate max-21inp within 
a ratio better than 1 /p'* . We use our results so as to clarify some aspects 
of Khot’s unique games eonjeeture. Namely, we show that for every e > 0 
it is NP-hard to approximate the value of unique games within a ratio 
of e. 



1 Introduction 

Systems of linear equations can be solved in polynomial time using Gaussian 
elimination. However, max-lin, the problem of finding a solution that maximizes 
the number of satisfied equations (in a system that is nonsatisfiable) is NP-hard. 
Here we consider the approximation ratios achievable for max-lin, where for a 
given algorithm this ratio is taken to be the number of equations satisfied by 
the algorithm divided by the number of equations satisfied by the best solution. 
In the following discussion we parameterize max-lin by two parameters, k and 
p, and denote the resulting problem as max-fclinp. Here, k is the number of 
variables per equation, and p is a prime, signifying that all computations are 
performed modulo p. 

For every system of linear equations modulo p, a simple greedy algorithm is 
guaranteed to satisfy at least a 1/p fraction of the equations. Hence for every k 
and p, max-fclinp can be approximated within a ratio of 1 /p. A celebrated result 
of Hastad [8] shows that for k = 3, this is essentially best possible. That is, for 
every prime p and for every e > 0, max-31inp is NP-hard to approximate within 
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a ratio of 1/p + e. In particular, max-31in2 is NP-hard to approximate within a 
ratio better than 1/2 + e. 

The situation for the case fc = 2 is different. Here there are approximation 
algorithms with approximation ratios significantly better than 1/p. Goemans 
and Williamson [7] gave an approximation algorithm based on semidefinite pro- 
gramming that approximates max-21in2 within a ratio of roughly 0.87856. This 
result was extended in [2, 9] to show that for every prime p there is some e that 
depends on p such that max-21inp can be approximated within a ratio better 
than (1 -I- e)/p. The value of e in these proofs converges to 0 as p grows. 

In terms of hardness results, Hastad [8] shows that max-21in2 is NP-hard to 
approximate within a ratio of 11/12 -|- e. In [2] it is shown that it is NP-hard to 
approximate max-21inp within a ratio of ||, || (-|-e, in all cases) for p = 3, 5, 7 
respectively, and within a ratio of 11/12 -|- e for all p > 11. This state of affair 
raises the following questions. 

1. The approximation ratios of known algorithms for max-21inp deteriorate and 
tend to 0 as p grows. The hardness results to not exclude approximation 
ratios as good as 11/12 for every p. Does, indeed, the best possible approx- 
imation ratio for max-21inp tend to 0 as p grows? This question was asked 
explicitly in [2]. In this paper we give a positive answer to this question. 

2. The hardness of approximation ratios for small values of p (p = 3,5,7) are 
not as strong as those for p = 2. Is it indeed easier to approximate max- 
21inp when p = 3, 5, 7 than it is when p = 2? In this paper we give a negative 
answer to this question. 

In the rest of the introduction we discuss our results in more detail. In partic- 
ular, we also show how our results are connected to the unique games conjecture 
of Khot [9] . 



1.1 Small Values of p 

A reduction between two optimization problems is approximation preserving if 
there is some constant c > 0 such that the optimum value of the target prob- 
lem is exactly c times the optimum value of the source problem. A polynomial 
time approximation preserving reduction from problem A to problem B can be 
used to transform any polynomial time approximation algorithm for problem B 
into a polynomial time approximation algorithm for problem A, with the same 
approximation ratio. 

Theorem 1. For every prime p > 3, there is a polynomial-time approximation- 
preserving reduction from max-2lin2 to max-2linp. 

Theorem 1 is proved by using local gadgets. Every equation modulo 2 is 
replaced by a collection of equations modulo p according to some fixed rule, in- 
dependently of all other equations. However, the analysis of why this reduction 
is approximation preserving is global (considering all equations simultaneously) 
rather than local (analyzing every gadget application separately). The global 
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nature of the analysis of our gadgets may be the reason why despite their sim- 
plicity, they have not been discovered earlier. In particular, Trevisan, Sorkin, 
Sudan and Williamson [12] present a systematic way (based on linear program- 
ming) for finding the optimal local gadgets in reductions, but their approach (as 
presented in [12]) is limited to gadgets whose analyses are local. 

Using the fact that it is NP-hard to approximate max-21in2 within a ratio 
strictly ofll/12-|-e. Theorem 1 implies: 

Corollary 1. For every e > 0 and prime p, it is NP-hard to approximate max- 
2linp within a ratio o/ll/12-|-e. 

The following question remains open. Let p and q be two arbitrary primes 
s.t. p > q. Observe that the discussion in the introduction implies that the 
approximation ratio for max-Sling is strictly better than that for max-31inp. Is 
the approximation ratio for max-21ing strictly better than that for max-21inp? 

1.2 Large Values of p 

When p is sufficiently large, the bounds implied by Theorem 1 can be greatly 
improved. 

Theorem 2. There is some 5 > 0 such that for every prime p, max-2linp is 
NP-hard to approximate within a ratio better than 1/p^- 

The proof of Theorem 2 is by a reduction from a problem that is sometimes 
called label cover [4], a notion recalled under different terminology in Theorem 3. 

1.3 The Unique Games Conjecture 

The unique games conjecture of Khot [9], if true, has far reaching consequences 
regarding approximation of optimization problems. In particular, Khot and 
Regev [10] show that it would imply that vertex cover is NP-hard to approximate 
within a ratio of 2 — e. We recall this conjecture. The notation and terminol- 
ogy that we use in our presentation were chosen so as to clarify the connection 
between unique games and the problem of max-21in. 

We denote a (one round, two provers) game by G = G{X,Y,A,B,Q,P). 
There are two finite sets of variables, X and Y, and two finite sets of values, A 
for X, and B for Y. There is a support set Q C V x T. In the most general form 
of games, there is a probability distribution tt defined over Q. In the context 
of the current paper, we may take tt to be uniform, and so as to simplify the 
presentation, we omit all references to tt. For every pair of variables (x,y) G Q, 
there is a polynomial time computable predicate Px,y{x,y). (More explicitly, 
the pair {x, y) plays two roles here, both to specify which predicate is being 
evaluated, and as variables in the predicate itself. Equivalently, we may think 
of there being only one predicate P over four variables, the first two being the 
names of x and y, and the last two being the values of x and y. The predicate 
is polynomial time computable in the sense that given values for x and y, it 
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can be evaluated in time polynomial in \X\ + |F|.) The goal of the game is to 
find two functions, Pi : X — > A and P 2 : Y — > B, that satisfy the maximum 
number of given predicates. (That is, for every (x,y) G Q, we check whether 
Px,y{Pi{x) , P 2 {y)) evaluates to true.) The value of a game G, denoted by w(G), 
is the maximum over the choices of Pi and P 2 of the number of predicates 
satisfied, divided by the normalizing factor |Q| (which is the total number of 
predicates). Hence 0 < oj{G) < 1. 

A game is said to have oneway uniqueness if for every {x, y) € Q and every a G 
A, there is exactly one b G B such that Px,y(a, b) holds. The following theorem 
is well known and has been used extensively in previous work on hardness of 
approximation . 

Theorem 3. For every e > 0 there are sets A and B with \A\ + \B\ < 

such that it is NP-hard to distinguish between oneway unique games G with 

w(G) = 1 and oneway unique games G with w(G) < e. 

Proof. For completeness, we sketch the proof (which can be regarded as folk- 
lore). The starting point is the fact that it is NP-hard to distinguish between 
satisfiable 3CNF formulas and those that are at most i5-satisfiable, for some 
i5 < 1 [5] . From this one can design a oneway unique game G in which X is the 
set of clauses, Y is the set of variables, and Q contains all clause-variable pairs 
where the variable appears in the particular clause. Every clause has 7 satisfying 
assignments. Encode these assignments as numbers in {1, ..,7} (where i encodes 
the ith satisfying assignment in lexicographic order). The set A is satisfying as- 
signments to clauses encoded as explained above, and the set B is assignments 
to single variables. Predicate Px^y(a, b) (where j/ is a variable in clause x) holds 
if the triple assignment a satisfies the clause x, and a and b are consistent in 
their assignment to y. oj{G) = 1 if the original 3CNF formula is satisfiable, and 
uj{G) = {2 + S)/3 if the original formula was only <5-satisfiable. Thereafter, the 
parallel repetition theorem of Raz [11] gives the desired result. ■ 

To avoid the possibility of misunderstanding, let us point out that in The- 
orem 3 (as well as elsewhere when we discuss games) we view the sets A and 
B as fixed for a whole class of games, whereas other parameters of the games 
(X,Y,Q,P) change within the class of games. Hence the notion of NP-hardness 
relates to algorithms whose running time is polynomial in |A| -|- |T|. 

A unique game has the oneway uniqueness property in both directions, 
namely, also for every b G B there is exactly one a G A such that Px^y{a,b) 
holds. The unique games conjecture of Khot [9] says that a weaker form of The- 
orem 3 holds also for unique games. 

Conjecture 1. (The unique games conjecture of Khot) For every 0 < e < 1/2 and 
every 0 < <5 < 1/2, there are sets A and B such that it is NP-hard to distinguish 
between unique games G with w(G) > 1 — e and unique games G with uj{G) < S. 

Equivalently, for every e > 0 and every 0 < 77 < 1, there are sets A and B 
such that it is NP-hard to distinguish between unique games G with w(G) > rj 
and unique games G with w(G) < e • 77 . 
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The condition 77 < 1 cannot be strengthened to 77 = 1 because there is a 
polynomial time algorithm that decides whether oj{G) = 1 for unique games. 

A weaker version of the unique games conjecture simply asks whether for 
every e, it is NP-hard to approximate u){G) for unique games within a ratio 
better than e. Technically, this weaker version replaces the condition “for every 
e > 0 and every 0 < 77 < 1” in Conjecture 1 by the weaker condition “for 
every e > 0 there is some 0 < 77 < 1 such that”. Khot observed that even this 
weaker form of the unique games conjecture was not known to be true, and asked 
whether it is true. We give a positive answer to this question. Our approach is 
explained in the following. 

Motivated by the unique games conjecture, we prove a stronger version of 
Theorem 2. Call a system of equations modulo p proper if it has the following 
properties: 

1. The set of variables can be partitioned into two disjoint sets, X and Y. 

2. Every linear equation contains exactly one variable from X and exactly one 
variable from Y (with nonzero coefficients!). 

3. For every pair of variables x € X and y G Y, there is at most one equation 
containing both x and y. 

Proper max-2linp is the problem of finding an assignment that maximizes 
the number of satisfied equations in a proper instance of 21inp. With some extra 
work, the proof of Theorem 2 can be extended to show that: 

Theorem 4. There is some 5 > 0 such that for every prime p, proper max-2linp 
is NP-hard to approximate within a ratio better than \jp^ ■ 

As a corollary, we have that unique games are hard to approximate within 
any e > 0 . 

Corollary 2. For every e > 0, there are some constant 0 < 77 < 1, prime p, 
and sets A, B with \A\ = \B\ = p, such that it is NP-hard to distinguish between 
unique games G with u){G) > rj and unique games G with uj{G) < e • 77 . 

Proof. A proper 21inp system S of equations is a unique game G. The sets A, Y 
of variables for S serve as the sets X, Y of variables for G. The set of values 
A, B for G corresponds to the set {0, 1, ... ,p — 1}. The support Q corresponds 
to all pairs of variables (x, y) that appear in some equation in S. As each pair 
(x,y) € Q appears in exactly one equation in S (because S is proper), this 
equation serves as the predicate P^^y for G. The uniqueness property follows 
from the fact that modulo p, every equation of the form cx = d (with nonzero 
coefficient c and arbitrary d) has a unique solution, and similarly for cy = d. 

Having established the fact that a proper 21inp system is a unique game, it 
remains to choose the parameters of the system. Choose p large enough such that 
1 /p^ in Theorem 4 is smaller than e. Then for proper max- 21 in 73 there is some 
constant 0 < 77 < 1 such that it is NP-hard to distinguish between instances 
in which an 77 -fraction of the equations are satisfied, and instances in which at 
most an e • 77 -fraction of the equations are satisfied. As desired. ■ 
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(Note: an explicit value for 77 can in principle be derived from the proof of 
Theorem 4, though at the moment there does not seem to be much interest in 
doing so). 

1.4 Equations over the Rationals 

In all our reductions, if one treats the system of equations as a system over the 
rationals (and then we call the underlying problem max 21in, omitting the p) 
or over the integers, rather than modulo p, the same hardness of approximation 
result follow. As the prime p increases, the approximation ratios proved hard in 
Theorem 2 tend to 0. One obtains: 

Theorem 5. For every e > 0, it is NP-hard to approximate max-2lin within a 
factor of e. 

Using our techniques, one can also prove hardness of approximation results 
for max-21in within ratios that tend to 0 as the size of the input instance increases 
(e.g., of the form e = for 7 < 1, where n is the number of equations in 

the system). However, these results are proved under the assumption that NP 
does not have quasi-polynomial time algorithms. It is an open question whether 
such results can be proved under the assumption that P yf NP. 

Theorem 5 and the discussion that follows should not be confused with hard- 
ness results appearing in [1,3], as there the number of variables per equation is 
not limited to 2. Additional discussion on the difference between 2 versus many 
variables per constraint can be found in [6]. 

2 Proofs 

We present here the proofs of our main theorems (except for those whose proof 
was already sketched in the introduction). 

2.1 Small Values of p 

We restate Theorem 1 in more detail, and provide the proof. 

Theorem 6. For prime p > 3 There is a polynomial time reduction transform- 
ing an instance E of max 2lin2 to an instance F of max 2linp s.t. one can satisfy 
k equations in E iff one can satisfy p • k equations in F. For p = 3, there is a 
polynomial time reduction transforming an instance E of max 2lin2 to an in- 
stance F of max 2lin3 s.t. one can satisfy k equations in E iff one can satisfy 
2k equations in F. 

Proof. Every equation in E is of the form x -\- y =2 0 or a; -I- y =2 1. Consider 
first the case in which p > 3. Transform an equation of the first form to the set 
of 2(p — 1) equations 

X =p iy 
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for i = 1, . . . ,p — 1 and 

x-l=pi{y- 1) 

for i = 1, ... ,p — 1. Transform an equation of the second form to the equations 

x-l=piy 

for i = 1, ... ,p — 1 and 

X =p i{y - 1) 

for i = — 1. Call such a group of equations a block. Assignments of 

0/1 values to x and y that satisfy an original equation will satisfy exactly p 
equations in the corresponding block. Assignments of 0/1 values which do not 
satisfy the original equation do not satisfy any equation in the corresponding 
block. Non-0 / 1 assignments to either x or y satisfy at most two equations in the 
corresponding block. 

This proves that if one can satisfy k equations in E, one can satisfy p ■ k 
equations in F. To see the converse look at a satisfying assignment a of F 
satisfying p ■ k equations. If all the variables receive 0/1 values the claim follows 
immediately (Give to all the variables in E the values given to them by u ). If not, 
call every variable not receiving a value from {0,1} misbehaved, and assign to all 
misbehaved variables independently at random 0 w.p. half and 1 w.p. half. Every 
block of equations containing a variable not set to values in {0, 1} will have p of 
its equations satisfied with probability 1/2 and zero of its equations satisfied with 
probability 1/2. Hence the expected number of satisfied equations in such a block 
is p/2. We conclude that there is a 0/1 assignment to the misbehaved variables 
such that if we have I blocks containing misbehaved variables the number of 
satisfied equations in those blocks will be at least pl/2. a satisfied at most 21 
equations in those I blocks. As p/2 > 2 (recall we assumed that p > 3) we get 
a 0/1 assignment satisfying more equations than a. In particular it satisfies at 
least p • k equations and we are done as we have seen that a 0/1 assignment 
satisfying p • k equations of F implies we can satisfy k equations in E. 

In case p = 3 transform an equation of the type x + y =2 0 to the 3 equations 

a: - y =3 0 



a: - 1 =3 2(y - 1) 

X =3 2y 

An equation of the type a; + y =3 1 will be transformed to the 3 equations 



X =3 2(1 - y) 



a; - 1 =3 y 
a: =3 1 - y 

One can check that assigning the variables 0/1 values which satisfy the original 
equation will satisfy two out of the three equations, assigning the variables 0/1 
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values which do not satisfy the original equation will satisfy zero out of the three 
equations and giving to at least one of the variables value 2 will satisfy at most 
one equation out of the 3. An analogue argument to the one given in the last 
paragraph shows that if we have an assignment satisfying 2k equations in the 
new system we have such an assignment with 0/1 values (in our random process 
every block containing a misbehaved variable will have an expected number of 
(2 + 0)/2 = 1 satisfied equations). The rest of the arguments are as in the above 
paragraphs. ■ 

2.2 Large Values of p 

Here we prove Theorem 2. 

Proof. The proof is by reduction from Theorem 3, and the reader is advised to 
recall the relevant notation for games. For simplicity in our presentation, rather 
than fix p in our proof and then figure out the relevant parameters for the game 
G, we shall fix parameters for the game G, and based on them choose p. It is 
not difficult to exchange the order of these choices, and we omit the details. 

Consider an arbitrary game G with oneway uniqueness and |A| > \B\ (as 
is indeed the case in the games appearing in the proof of Theorem 3), and let 
p = 6>(|A|/e), for e as in Theorem 3. Let a be an arbitrary one to one mapping 
from A to Zp and r be an arbitrary one to one mapping from B to Zp. Thus 
from now on we look at elements of A and B as elements of Zp 

For every pair (x, y) G Q, replace the predicate Px,y by a block of \A\{p — 1) 
equations as follows. For every a G A the block contains p — 1 equations x — a = 
i{y — b), where 1 < i < p — 1, and b € B is the unique value s.t. Px^y{a, b) = 1. 
Looking at a group of p — 1 equations at a specific a, we see that an assignment 
that zeroes {x — a) and (y — b) will satisfy all p — 1 equations whereas any other 
assignment will satisfy at most one equation. 

For any assignment assigning c to a: and dto y where (x, y) G Q we have that 
if Px,y{o'~^ (c) , T~^ (d)) = 1, at least p — 1 equations will be satisfied out of the 
block corresponding to (x,y). If on the other hand, Px,y{(i~^{c),T~^{d)) = 0, at 
most I A| equations would be satisfied in the {x, y) block. We remark that in any 
event we cannot satisfy more that p — I + |A| — I equations per block. 

Since we have a block of equations for every pair (x, y) G Q, we get that if 
uj{G) = I then we could satisfy at least (p — I)|Q| equations. 

Assume now that tt is an assignment that satisfies more than ((I — e)|A| + 
e(p— I + |A| — 1))|(5| equations, where e is as in Theorem 3. Since tt cannot satisfy 
more than p — 1 + | A| — I equations per block, it must hold that the fraction of 
blocks in which tt satisfies more than |A| equations is strictly larger than e. Hence 
by the discussion above it follows that for G, the assignment Pi{x) = a~^{7r{x)) 
for all cc G A and P 2 {y) = r“^(7r(p)) for all p G V shows that w(G) > e. Thus 
we get a hardness of (recall that \A\ < p) 

((l-e)|A| + e(p-I+|A|-I))|Q| ^ |A| 

(P-I)IQI -p-1 

As p = 0(|A|/e) and |A| = (I/e)‘^(^), the proof follows. ■ 
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2.3 Proper Max-21inp 

The proof of Theorem 2 gives a system of equations arranged in blocks. To every 
pair of variables (x, y) € Q corresponds one block of exactly |^|(p — 1) equations. 
In what follows, the parameters q and m will be used to denote q = \A\(p — 1) 
and m = 40qp^. In our proof that proper max-21inp is hard to approximate, we 
shall make use of a notion that we call balanced coloring. 

Definition 1. Let c be an edge coloring with q colors of the complete bipartite 
graph Km,m- Call a vertex-induced subgraph G = (V, E) big if \E\ > mf /p^ . We 
say c is balanced if the following two conditions hold: 

1. Every color is assigned to at least j2q edges. 

2. For every big induced subgraph G = {V, E), every color is assigned to at most 
^{\E\/q) edges in E. 

Lemma 1. Balanced eolorings as defined above, exist. 

Proof. We give an existence proof using the probabilistic method. Color all the 
edges of Km,m with q colors by assigning independently at random to each edge 
a color between 1 and q (each color is chosen with probability |). Let G = {V, E) 
a fixed big induced subgraph of Km,m- The probability that a given color will 
occur more than ^\E\/q times or less than ^\E\/q in this induced subgraph is 
by the Chernoff Bound at most 

2 • (l/e)0-38'il^l/9 < 2 • 

The number of induced subgraphs of Km,m is bounded by 2^™. Taking the 
union bound on all colors and all induced subgraphs we get that the probability 
some big subgraph will not have the desired property is bounded by 

. 2'°89+2™+i < 

As we took m to equal AQp^q the exponent of the above expression is positive. 
Hence, with positive probability we get a balanced coloring and the result follows. 



In our proofs we will need balanced colorings when the parameters m and 
q can be regarded as constants (even though they are “large” constants), and 
hence a balanced coloring can be found in constant time by exhaustive search. 

Let E be an instance of max 21inp with the block structure as described 
above. That is, every block has q equations and all the equations in a block 
contain the same two variables (and every equation contains an X variable and 
a Y variable). For this reason, this 21inp system is not proper, in the sense defined 
in Section 1.3. We reduce the improper (by improper we mean an instance that 
is not proper) instance of 21inp (that we call E) to a proper instance of 21inp 
(that we call E'). 

The set of variables for E' will be m times as large as that of E. Every variable 
X G X for E will be “replaced” by m variables Xi, ... , Xm in E' . Likewise, y GY 
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will be replaced by j/i, . . . , j/m- As to the equations, every block of q equations 
in E is replaced by a block of m? equations in E' . This is done as follows. Let x 
and y be the two variables occurring in a block of E. Assign to every equation 

in the the block a different color from 1,2 ,q (in arbitrary order). Color the 

edges of Km,m with a balanced coloring with q colors. If in this coloring the color 
of the edge between i and j is I, put Xi and yj in equation 1. We remark that this 
coloring is used simultaneously in all the blocks of equations of |if'| and that the 
number of blocks in E' is |(5|, the same as the number of blocks in E. It is not 
hard to see that E' is indeed proper. 

Denote by X' the set of all the x variables in E' and by Y' , all y variables 
in E'. Say a set i? = S' x T C X' x F' is a rectangle if both S and T are subsets 
of the variables in the same block of if'. A rectangle is big if it contains at least 
a 1/p^ fraction of the equations in a block. 

Let r be an arbitrary assignment to the variables of E' . For any block of 
E' , let x\, . . . ,Xm,yi, ■ ■ ■ ,ym be its variables. Assignment r partitions both 
x\, , Xm and j/i , . . . , j/m to at most p nonempty sets Sk , where Xi belongs to 

the Sj if T(xi) is the jth value from the field and similarly for Tj with the y vari- 
ables. By the properties of balanced coloring, in every big rectangle R= Si xTn 
any color in R appears t\R\/q times where 1/2 < t < 3/2. (\R\ = |S'/| x |T„|). 

The proof of Theorem 4 now follows from the combination of Theorem 2 and 
the following lemma. 

Lemma 2. Let rj denote the optimum} of the improper 2linp system E with 
block structure as described above. Then rj' , the optimum of the proper system 
E' resulting from the reduction described above, satisfies rj/2 < rj' < 3rj. 

Proof. Let ct be an assignment that satisfies rjq\Q\ equations in E. Define an 
assignment t to the variables of E' as follows: for each tuple of m variables 
zi,...,Zm corresponding to a variable z of the original system put t{z\) = 
t{z 2 ) = ... = T{zm) = cf{z). By property 1 of balanced coloring we have that t 
satisfies at least ^rjmf\Q\ equations in the new system. Hence rj' > rj/2. 

We now prove that 77 ' < 3r], or rather, that 77 > T 7 '/ 3 . Let r be an optimal 
assignment to the variables of the new system, satisfying r]'mf\Q\ equations. 
Index the blocks with numbers between 1 and \Q\. Let k be an arbitrary block 
of equations in E' , and let x and y be the variables in the corresponding block of 
E. As explained above, the assignment r partitions the block k into rectangles 
Si,k X Tj^k {i} Y i, j < p — 1 ), where x variables are in Si^k if they are given the 
value i by the assignment and similarly for y variables. For any big rectangle 
define Cij^k to be the fraction of equations of Si^k x Tj^k satisfied by t. The 
number of equations in rectangles that are not big is bounded by ^ • m^. 
Thus, the number of equations satisfied by r in E' is at most 

IQI 

777 \Q\/p+ {\Si^k\\Tj^k\Cij,k) (1) 

k—1 'is big 

^ By optimum we mean the maximum fraction of equations that can be satisfied by 
an assignment. 
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Construct an assignment cr to the variables of i? as follows: choose k and I 
uniformly at random from [1, ..m]. For every x € X, let a{x) = T{xk) (where Xk 
is the /cth variable replacing x), and for every y gY, let a{y) = T{yi). Let Z be 
the following random variable: Z counts the number of equations of E satisfied 
by a. Its expectation satisfies: 

E{Z)>Y. E p) 

k=l is big 

The above inequality follows from property 2 of balanced colorings. Namely, 
in a big rectangle R no color appears on a fraction of more than X of the 
edges (equations). In a block of E, the same color appears on exactly l/q of the 
equations. Hence in E the fraction of satisfied equations shrinks by a factor no 
worse than | compared to R. 

There must be some assignment to E that achieves at least the expectation of 
Z. Comparing equations (1) and (2), this implies that rj > — 1/p). Moreover, 

77 > 1 /p, as in every system of equations modulo p one can satisfy a 1 /p fraction 
of the equations. Hence y > y' / 5, as desired. ■ 
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Abstract. Utility functions satisfying gross substitutability have been 
studied extensively in the economics literature [1, 11, 12] and recently, the 
importance of this property has been recognized in the design of combi- 
natorial polynomial time market equilibrium algorithms [8]. This natu- 
rally raises the following question: is it possible to design a combinato- 
rial polynomial time algorithm for this general class of utility functions? 
We partially answer this question by giving an algorithm for separable, 
differentiable, concave utility functions satisfying gross substitutes. Our 
algorithm uses the auction based approach of [10]. 

We also outline an extension of our method to the Walrasian model. 



1 Introduction 

The recent papers of Papadimitriou [16] and Deng, Papadimitriou and Safra 
[5], which raised the issue of efficient computability of market equilibria, have 
resulted in considerable activity on this topic. Some of the algorithms are based 
on solving nonlinear convex programs [14,18], following the classical approach 
of Eisenberg and Gale [9], while others are combinatorial [6, 10, 13, 7, 8, 17]. The 
latter algorithms are dominated by two main techniques: primal-dual-type algo- 
rithms which were initiated by Devanur, Papadimitriou, Saberi and Vazirani [6], 
and auction-based algorithms introduced by Garg and Kapoor [10]. 

Both these papers deal with linear utility functions. Moreover, both start 
with very low prices and monotonically raise them until equilibrium prices are 
reached. The usefulness of such a scheme was clarified in [8]: prices play the 
role of dual variables in primal-dual algorithms, and almost all such algorithms 
known in the areas of exact and approximation algorithms work by greedily rais- 
ing dual variables - arranging anything more sophisticated is extremely difficult. 
Furthermore, linear utility functions satisfy gross substitutability, i.e., increas- 
ing the price of one good cannot decrease the demand for another. Hence, for 
such utility functions, monotonically raising prices suffices. In contrast concave, 
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and even piecewise-linear and concave, utility functions do not satisfy gross sub- 
stitutability, and designing market equilibrium algorithms for them remains an 
outstanding open problem. 

Linear utility functions have been generalized to spending constraint util- 
ity functions, which also satisfy gross substitutability, in [17]. Building on the 
algorithm of [6], a primal-dual-type polynomial time algorithm for spending 
constraint step utility functions has been obtained in [17]. The importance 
of gross substitutability has been recognized in the economics literature, and 
there have been attempts at designing algorithms for this general class of util- 
ity functions [1,11]; however, to our knowledge no such polynomial time al- 
gorithms are known. In this paper, we use the auction-based approach of [10] 
to give polynomial time approximation algorithms for computing equilibria for 
the general class of additively separable, differentiable, concave utility func- 
tions satisfying gross substitutability. Our algorithm achieves a complexity of 
0{{E/e) log(l/e) \og{{evVmax) / { eminVmin)) log to) where E is the number of non- 
zero utilities Uij, Vij{xij) is the slope of Uij w.r.t. Xij and Cj are the endowments 
(in terms of money). In the complexity description, Cmm = min^ Cj, e = 
^Vmaxlvmin is the ratio of the largest slope to the least slope (we assume that 
this ratio is bounded), v = ^ the tolerance parameter to 

which equilibrium prices are computed. This algorithm is faster by a factor of 
0{n) as compared to the auction algorithm for the Arrow-Debreu model [10] and 
also extends the class of utility functions for which approximate market clearing 
can be achieved. 

Interestingly, while the algorithm’s framework remains as simple as in [10], 
the proof of correctness and convergence are complicated by the general nature 
of the utility functions. We are able to, in this paper, resolve the convergence to 
the equilibrium prices via a monotone change in prices for separable increasing 
functions satisfying gross-substitution and concavity. The problem of comput- 
ing equilibrium in the more general class of (non-separable) increasing concave 
functions satisfying gross substitutability is a challenging one. 

In Section 2 we define the market model and provide a characterization of 
gross substitutable functions. In Section 3 we outline our algorithm and prove 
correctness and the complexity bounds. Finally, we outline (Section 4) an exten- 
sion to the Arrow-Debreu model. 

2 Market Model 

Consider a market consisting of a set B oi n buyers and a set A of m divisible 
goods. Buyer i has, initially, an amount of money equal to e^. The amount of 
good j available in the market is aj . We will assume that the utilities are additive 
and separable. Buyer i has a utility function, Ui{Xi) = EjUijixij) where Xij is 
the amount of allocation of good j to buyer i, Aj = {xn,Xi 2 ■ ■ ■ Xim) represents 
the current allocation vector of goods to buyer i and Uij(xij) : R+ R+ is the 
utility function of item j to buyer i. We assume that Uij is non-negative, strictly 
increasing, differentiable, and concave in the range [0,0^]. Let Vij represent the 
first derivative of Ui w.r.t. Xij (which is well defined). Assume that the buyers 
have no utility for money, however they use their money to purchase the goods. 
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Given prices P = {pi,P 2 , • • ■ ,Pm} of these m goods, a buyer uses its money to 
purchase goods that maximize its total utility subject to its budget constraint. 
Thus a buyer i will choose an allocation Xi that solves the following buyer 
program Bi{P)\ 



Maximize : Uij{xij) 


(1) 






Subject to: ^ijPj ^ 


(2) 


l<^<m 




Vj : Xij > 0 


(3) 


This defines a family of programs, each program parameterized by a fixed price. 
Since Uij is concave for all i and j, the theory of duality (Kuhn- Tucker condi- 


tions) can be used to give the following necessary and 
optimality for a given price vector P: 


sufficient conditions for 


^ijPj ~ 


(4) 






Vj : aiPj > Vij(xij) 


(5) 


Vj : Xij ^ 0 CXiPj — Vij{Xij^ 


(6) 


ai > 0,Vj : Xij > 0 


(7) 



In the above conditions, is the marginal utility per unit of money, also referred 
to as marginal bang per buck, for buyer i. If Xij > 0, then at Xij the marginal 
bang per buck for buyer i from good j ( Vij{xij)/pj) is exactly aj. Otherwise, 
Vij{Q)/pj < ai- We say that the pair {X, P), X = {Xx,X 2 ■ ■ ■ Xn) forms a market 
equilibrium if (a) the vector Xi G i?" solves the problem Bi{P) for user i and 
(b) there is neither a surplus or a deficiency of any good i.e., 

Vj : ^ ' Xij = aj (8) 

l<z<n 

The prices P are called market clearing prices and the allocation X is called 
the equilibrium allocation at price P. 

The equations (8) and (4) imply that all the goods are sold and all the buyers 
have exhausted their budget. Equations (5) and (6) imply that (a) that every 
buyer has the same marginal utility per unit price on the goods it gets and (b) 
every good that a buyer is not allocated provides less marginal utility. 



2.1 Gross Substitutes 

Gross substitutes is a well-studied property that has useful economic interpreta- 
tions. Goods are said to be gross substitutes for a buyer iff increasing the price 
of a good does not decrease the buyer’s demand for other goods. Similarly, goods 
in an economy are said to be gross substitutes iff increasing the price of a good 
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does not decrease the total demand of other goods. Clearly, if the goods are gross 
substitute for every buyer, they are gross substitutes in the economy. 

We now give a formal definition of gross substitute in our model. Consider 
the buyer maximization problem Bi{P). Let Si{P) C be the set of optimal 
solutions of the program Bi{P). Consider another price vector P' > P. Goods 
are gross substitutes for buyer i if and only if for all Xi £ S'i(P) there exists 
X- £ Si(P') such that pj = p' Xij < x'^. 

Note that Uij is continuous, concave and differentiable for all i and j and 
Vij{x) = -^Uij{x). Since Uij is concave, Vij is a non-increasing function. The fol- 
lowing result characterizes the class of separable concave gross substitute utility 
functions. 

Lemma 1. Goods are gross substitutes for buyer i if and only if for all j, yvij{y) 
is a non- decreasing function of the scalar y. 

Proof. To prove the converse part (which is critical to the algorithm in Section 3), 
assume that there are scalars y and y' such that y' < y and y'vij{y') > yvij{y). 
Choose a price P and an optimal solution Xi of Bi{P) such that Xij = y. Let 
ai be the optimal dual solution of Bi{P). Construct a corresponding P' and X[ 
such that x'^f. = x^k, p'k = Pk for all k yf j, a;C = y' and p' = pjVij{xG)/vij{xij). 
Now, 

x'.jP'j = x[jPj Vij {xG )/vy (xy ) 

> PjXijVij{Xij) /vij{xij) 

= XijPj 

So, the solution Xl satisfies (11) and (10) for price P', but ^J^ixGpb > 
Yl^=iXijPj = Ci. Therefore, the optimal dual solution a' of Bi(P') will sat- 
isfy a' > ai. Therefore, the optimal solution X” of Bi{P) will have x'). < xC 
for all k ^ j, such that Xik >0. Hence the goods will not be gross substitute for 
buyer i. 

We next show that yvij{y) is non-decreasing then the goods satisfy gross- 
substitutability. Consider an optimal solution Xi £ Si{P). The dual of the 
program Bi{P) gives the following necessary and sufficient conditions for the 
optimality of Xi. 



^ ' XijPj — Si (9) 

i=i 

Vj . Xij ^ 0 Vij{Xij^ — aiPj (11^) 

yj ■ a^pj >Vij{xij) (11) 

ai > 0 , Xij > 0 

If Xij > 0, equation (10) gives XijPj = XijVijjai. Consider P' > P. For 
this price vector we construct a feasible solution Xl as follows: If p' = pj then 
x'ij = Xij , VL Alternately, if Vij (0) < a^p' then set xC to zero, else choose xC such 
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that Vij{x[j) = aip'j. By definition, the solution X' satisfies the complementary 
slackness conditions (10). Since P' > P, also satisfies (11). Also since Vij is 
a non-increasing function p' > pj xG < Xij. Now, 

^ j Ol,i 

= XijPj 

The above equations give 

m m 

= e* 

i=i j=^^ 

Note that Uij is concave for all j. Therefore, there is an optimal solution X” of 
the program Bi(P') such that X-' > X'. ^From the definition of X' if pj = p'- 
then xG = Xij. Therefore pj = p' x". > Xij where Xi is an optimal solution 
of Bi{P) and X” is a corresponding optimal solution of Bi{P'). 

3 An Auction Algorithm for Market Clearing 

We now present an ascending price algorithm for discovering the market clearing 
prices approximately. The algorithm starts with a low price and an initial allo- 
cation Xi for all buyers i, such that all the goods are completely allocated and 
optimal allocation of buyers dominate their current allocation. Now the prices of 
over-demanded items are raised slowly and the current allocation is recomputed, 
until no item is over-demanded. This approach has a similarity with the Hun- 
garian method of Kuhn [15] for the assignment problem. Unlike the Hungarian 
method which raises the price of all the goods in a minimal over-demanded set 
by a specific amount, our algorithm raises the price of one good at a time by a 
fixed multiplicative factor (1-l-e), where e > 0 is a small quantity suitably chosen 
at the beginning of the algorithm. This algorithm has an auction interpretation, 
where traders outbid each other to acquire goods of their choice by submitting a 
bid that is a factor (1 -I- e) of the current winning bid. Prior to this, auction algo- 
rithms have been proposed for maximum weight matching in bipartite graphs, 
network flow problems and market clearing with linear utilities [4], [3], [2], [10]. 

We discuss the algorithm which is formally presented in Figure 1. In proce- 
dure initialize (Figure 1) all the items are allocated to the first buyer. Prices 
are initialized such that (a) the buyer’s money is exhausted and (b) the buyer’s 
allocation is optimal at the initial prices. An initial assignment of dual variables 
ai is also required. Instead of ai we maintain aij, a separate dual variable for 
each item, for computational ease. Finally we will relate ai to aij. 

It is easy to verify that the prices set in procedure initialize exhausts the 
budget of the first buyer. Since vij{aj) is assumed to be strictly positive^, pj > 0 
for all j. Therefore the initial value of aij is well-defined for all i and j. 

^ This is not a necessary assumption. It is made for simplicity of the presentation. A 
weaker assumption would be that every item j has a buyer i such that Vij{aj) > 0. 
The initial allocation may still be found that satisfies the desired properties. 
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Also note that jpj = a\j = a\ for all j. Hence the initial allocation 

maximizes the utility of the first buyer. 

The goods are allocated at two prices, pj and pj / (1+e) . The allocation of good 
j to buyer i at price pj is represented by hij and the allocation at price pij /{1 + e) 
is represented by yij. The total allocation of good j to buyer i is given by 
Xij = hij+Uij . Define the surplus of a buyer i asvi = '^j^i{hijPj + yijPj / {1 + e)). 
Define the total surplus in the system as r = Yll/=i D- 

The auction algorithm main (Figure 1) begins with a buyer i who has signifi- 
cant surplus (more than ee^) and tries to acquire items, with utility per unit price 
more than the current utility per unit price. It outbids other buyers by acquiring 
items at a higher prices. It raises the prices by a factor (I -|- e) if needed. This 
process continues till total surplus in the economy becomes sufficiently small. 

The algorithm maintains the following invariants: (ll) items are fully sold, 
( 12 ) buyers do not exceed their budget, (l3, 14) after completely exhausting its 
surplus a buyer’s utility is close to its optimal utility at the current prices, (l5) 
prices do not fall and (l6) total surplus money in the economy does not increase. 
Figure 2 lists these invariants formally. 

It is easy to check that all the invariants (ll through 16) are satisfied after 
the initialization (i.e. after procedure initialize has been called). 

The allocation Xij is modified only in procedure outbid. However, the mod- 
ifications leave the sum unchanged. Therefore the invariant II is 

satisfied throughout the algorithm. 

For invariant 12, it is sufficient to show that > 0 for all i. Vi is reduced 
only in procedure outbid. In this procedure, the variable is chosen such that 
Ti does not become negative and hence 12 remains satisfied. 

For invariant 16, note that the only steps that change r are in procedure 
outbid. In these steps, r is reduced by et. Hence 16 is satisfied in the algorithm. 
The invariant 15 is trivially satisfied 

We now show that invariants 13 and 14 are satisfied by the algorithm. 

Lemma 2. During the approximate auetion algorithm the invariants 13 and 14 
are always satisfied. 

Proof. The invariants are true initially for all the buyers. We first show invariant 
13. Note that when Oy is modified in algorithm main after calling outbid, the 
invariant is satisfied. Since pj never decreases, the invariant remains satisfied 
whenever pj changes. When Xij is reduced, Vij{xij) increases causing a potential 
violation of the invariant. In this case, the inner while loop of the algorithm will 
be executed. We argue that when the inner loop ends UijPj = Vij{xij) for all i,j. 

To prove this, consider the time instant z when good j was acquired by 
buyer i at price pj. Let a be the quantity of good j acquired by buyer i. Now, 
aijPj = Vij{a). Assume that the amount of good j currently acquired by buyer i 
isb < a. Let the current price of j be pi . Choose c such that Vij (c) /pi = Vij (a) /pj . 
It is always possible to do so since % (6) > ceijp/ > CiijPj = Vij{a). Since pi > pj, 
c < a. Now, cpl = cpjVij{c)/vij{a). From the assumption that goods are gross 
substitutes and using Lemma I we have cUy(c) < avij{a). Therefore cpl < apj. 
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procedure initialize 
: hij = 0 

Vi 7^ l,Vj : Vij = 0 

Vj : yij = 

Vi : Pj = vij(aj)/ai 

Vi 7^ 1 : = Vij{0)/pj ; n = a 

Vi 7^ l,Vi : Qij = Vij{xij)lpj 

ri — 0 

end procedure initialize 

algorithm main 

initialize 
while 3i : ri > ed 

while (n > 0) and Oj : aijpj < Vij{xij)) 

if 3k : Hkj > 0 then outbidCi, k, j, aij) 
else raise_price(i) 
end while 
i = arg maxi otu 
if 3k : ykj > 0 

outbidCi, k, j, Oij /{I + (-)') 

Oij = Vij{Xij)/pj 
else raise_price (i) 
end while 

end algorithm main 

procedure raise_price(j) 

Vi I yij ~ hij 5 hij = 0 j 

Pi = (1 + e)Pi 

end procedure raise_price 

procedure outbidCi, k, j, a) 

ii = ykj 
t 2 = n/pj 

if (vij{aj) > apj) then ts = aj 
else ts = min S : Vij{xij + 5) = apj 
t = min(fi,f2,i3) 

hij — hij -\- 1 
Vi = Vi - tpj 

ykj = ykj — t 

rk=rk + tpj/{l + t) 

end procedure outbid 



Fig. 1. The auction algorithm 



Therefore the amount of money needed to be spent on j to ensure Vjj {c)p'j = ocij 
is p'jC which is no more than Pja] the amount of money spent on j before Xjj 
was reduced. Hence, when the inner loop ends ajjp'j > Vij{xjj) for all i and j. 
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j- 




12 


V 


i: 


Y.j < ei 


13 


V 


i: 


Ci = 0 => aijPj > Vij{xij) 


14 


V 


fj 


Xij > 0 => (1 + e)vij{xij) > aijPj 


15 


V 


j- 


Pj does not fall 


16 






r does not increase 



Fig. 2. The invariants in the auction algorithm 



The invariant 14 is satisfied after initialization. Whenever is changed in 
main Vij {xij ) = aijPj . Therefore, if Xij is reduced, 14 remains satisfied. Xij may 
be increased by a call to outbid in the inner loop. However, the parameter 
in outbid ensures that the variable is chosen such that aijPj < Vij{xij). 
Moreover, if Xij > 0 at the exit of the inner loop, then aijPj = Vij{xij). So, 
if pj is raised by a factor (1 -|- e), 14 will still be satisfied. If pj rises by more 
than the factor 1 -I- e, Xij will be set to zero and when Xij is increased again 

remain satisfied in the algorithm. 

The algorithm ends when the surplus with each buyer is small, i.e. ri < eCj. At 
this stage the market-clearing conditions (4) and (6) are satisfied approximately 
and (5) and (8) are satisfied exactly. 



3.1 Convergence of the Algorithm 

In order to provide efficient convergence, bidding is organized in rounds. In each 
round every buyer (i) is picked once and reduces his surplus to 0, i.e. = 0. 
The following lemma proves a bound on the surplus reduction in each round. 

Lemma 3. In every round the total unspent money r = X)r=i decreases by a 
factor of {1 + e). 

Proof. The value of r is decreased in procedure outbid by tepj. Buyer i bids 
until Vi = 0. WLOG assume that these bids are of amounts ti,t 2 , ■ ■ ■ ,tk on 
items 1, 2, . . . , fc at prices Pi,p 2 , ■ ■ ■ ,Pk- Now we have: 

k 

+ e)pi = n 

1^1 



Reduction in r is given by 



Ar = 






Bidding by buyer i can only increase r^ for another buyer k. Therefore the total 
reduction in r in one round is given by: 



Ar>Y, 

2=1 



l + € 



e 

r 

1 + e 
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The new value of unspent money r' after every round is related to its old value 
r as: r = 

Let emin = mini and e = If ^ < ^^min then no buyer has signifi- 

cant money left. Therefore the algorithm is guaranteed to terminate in k rounds 
where k = log(^j-^)/log(l -I- e). 

Let E be the number of non-zero utilities. We first bound the number of calls 
to procedures raise_price and outbid. 

The price of any item is bounded by: 



(aj) 



ei 



At every step, the price is raised by a factor (1 -|- e), therefore the total number 
of calls to raise_price is bounded by: 



1 ev 

-log( 



) 



where v = = min* e* and = minyUifc(afc) . We will 

assume that the ratio of v and Vmin is bounded. If not, i.e. if Vmin is zero then 
we can perturb the utility function Uij{xij) by addition of the term exij such 
that the derivative is at least e. Furthermore, the derivative is bounded above 
by Vik{ak). 

In a call to outbid, one of the following four events can occur: (a) ykj becomes 
zero for some k, (b) becomes zero, (c) reduces by a factor of (1 -I- e) and 
(d) Vij(xij) reaches in the inner while loop of algorithm main. 

Events of type (a) are charged to calls to procedure raise_price and events 
of type (b) and (d) are charged to the current round of bidding. Events of type 
(c) are charged to a reduction in . 

Note that for every price rise of item j, the number of events of type (a) is 
bounded by the number of buyers having non-zero utilities on item j. Thus the 
total number of type (a) events is bounded by E times the maximum possible 
number of price rises for any given item. 

It is easy to see that number of type (b) events is exactly equal to n in 
every round of bidding. At every type (c) event, is reduced by a factor of 
(H-e). Since = Vtj{xij)jpj , its value varies from Vij{aj)/pm,ax to Vij{0)jpmin, 
where Pmin and Pmax are the minimum and maximum values of pj , respectively. 
In every round only one event of type (d) occurs for each buyer. 

During the algorithm evaluation of arg max/ an can be done efficiently by 
maintaining n heaps, one for each buyer. The heap of a buyer contains the 
goods with non-zero utilities sorted in decreasing order of an . Updates requires 
O(logn) steps. 

This gives us the following time complexity of the algorithm: 

Theorem 1. The auction algorithm terminates in 
0{{E/e)log{{evVjnax)/{eejntnVmin))'^ogn) steps. 

where Vmax = max^ fy(0). 
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4 Extension to the Arrow-Debreu Model 

The algorithm for the Fisher model described in the previous section can be 
extended to the general Walrasian model, also called the Arrow-Debreu model, 
where there is no demarcation between buyers and sellers, and all agents come 
to the market with an initial endowment of goods. 

In order to extend to this case, we first note that under the assumption that 
goods satisfy gross substitutability, a version of Lemma 1 still holds, i.e., xvij{x) 
is a non-increasing function with respect to buyer z if a: > aij . 

The algorithm is modified to compute surplus available with each of the 
traders based on the endowment of the trader. In each phase of the algorithm, 
a trader with surplus is picked. Firstly, items that do not satisfy the invariance 

CtiPj > Vij{x) 

are acquired by being outbid at the current price, failing which the price of the 
chosen item is raised. Gross substitutability ensures that the invariance would 
be satisfied when the surplus is exhausted. Alternately, if every item satisfies 
the invariance w.r.t. the chosen trader with surplus, the trader bids on other 
items which may either be available or whose price may have to be raised. The 
bidding for items is performed in rounds, with each trader being considered at 
least once in a round and exhausting his surplus, to ensure fast convergence. A 
detailed analysis of the efficient convergence of this bidding process is similar to 
that provided in the paper of Garg and Kapoor[10] for the linear case. 

5 Conclusions 

It would be of considerable interest to extend the class of utility functions for 
which the market equilibrium problem is solvable via the auction method. In 
particular, can this approach be extended to non-separable functions? 
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Abstract. We consider a single source network design problem from 
a game-theoretic perspective. Gupta, Kumar and Roughgarden {Proc. 
35th Annual ACM STOC, pages 365-372, 2003) developed a simple 
method for single source rent-or-buy problem that also yields the best- 
known approximation ratio for the problem. We show how to use a vari- 
ant of this method to develop an approximately budget-balanced and 
group strategyproof cost-sharing method for the problem. 

The novelty of our approach stems from our obtaining the cost-sharing 
methods for the rent-or-buy problem by carefully combining cost-shares 
for the simpler problem Steiner tree problem; we feel that this idea may 
have wider implications. Our algorithm is conceptually simpler than the 
previous such cost-sharing method due to Pal and Tardos {Proc. 44^h An- 
nual FOG'S, pages 584-593, 2003), and has a much improved approxima- 
tion factor of 4.6 (over the previously known factor of 15). 



1 Introduction 

This paper studies the problem of giving good cost-sharing mechanisms for a 
single source network design problem. Imagine a general network design problem, 
where the participants (or agents) want to build a network connecting them to 
a common source (a server); however, they are autonomous and behave in a 
selfish (but non-malicious) fashion. Informally, a cost-sharing mechanism builds 
a network, and allocates the cost incurred among the agents, so that no group 
of agents is charged too much, thus precluding the possibility of their being 
unhappy and trying to secede from the system. 

The type of problem we consider is where we are given an undirected graph 
G = {V,E), and a set of demands D C V that want to connect to a common 

* Research done in part during the IMA workshop on Network Management and De- 
sign at the University of Minnesota, April 2003. The research of the second author 
was supported in part by the National Science Foundation under Grant No. 0208005, 
and that of the third author in part by ONR grant N00014-98- 1-0589, and NSF grants 
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source r. A cost-sharing method is an algorithm that builds a cheap network, 
and also specifies what portion of its cost is paid by which of the participants 
in the network; this is done in a manner that ensures that the cost paid by any 
subset of the participants is fair. 

Given an instance of a network design game specified by a set of demands D, a 
cost-sharing scheme for the problem is simply a function ^(Z?, i) with ^{D, z) = 0 
for i ^ D. For a set D of demands we use OPT(Zl) to denote the minimum cost 
network serving the users in D, while Alg(£>) denotes the cost, or expected cost, 
of the network computed by a given algorithm. There are three properties that 
we will be concerned with: 

1. (/3-approximate budget-balance) For a set of demands D, we require 
that 

OPTp)>E.ez5C(Ai)>Alg(Z?)//3 

for some given parameter /3 > 1. Equivalently, by multiplying the shares 
up by (3, we could require that the cost-shares are at least the total cost 
of a solution found, but do not exceed j3 OPT(Z?). If /3 = 1, we call the 
cost-sharing budget-balanced. 

2. (fairness) For any AC D, < OPT(A). I.e., the cost paid by 

any subset of people should not exceed the optimal cost of connecting them 
alone and hence they have no incentive to secede. 

3. (cross-monotonicity) For any A C D, and demand i G A we require that 

i) < ^{A, i). I.e., the cost-share of any demand should not go up due to 
other demands entering the system. This property is also known under the 
name of population- monotone. 

Cross-monotonicity is a key ingredient used in solving the following type of 
mechanism design problems: consider the network design problem with a set of 
demand nodes D, with each user (or demand) d having an associated utility 
Ud- Since the users have limited utilities, the service provider has to now decide 
which subset of customers it must serve, in addition to designing the network 
and deciding how to share the cost between the served customers. A mechanism 
for solving this problem is called group strategyproof if no subset of users has 
an incentive to deviate from the protocol (e.g., by misreporting their utility) in 
the hope of improving the outcome for themselves (e.g., receiving the service 
at a cheaper cost). Moulin and Shenker [6] show that having a cross- monotone 
cost-sharing method for a problem naturally gives rise to a group strategyproof 
mechanism for the problem in the following way. We start with all the customers; 
if there is some customer whose cost share (computed w.r.t. the current set of 
customers) exceeds its utility, we drop it from the set, recompute cost shares 
and repeat. At the end, we are left with the desired set of customers and their 
cost-shares. 

As an example of cross-monotonicity, let us consider the Minimum Spanning 
Tree game on the complete graph G = (V,E) with edge weights, given by a set 
of players D and a root r ^ D; the objective is to find the cheapest tree MST(£)) 
spanning D and r. It must be emphasized that this game does not allow the 
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use of Steiner vertices in MST(_D), and r is not a player, and hence should have 
no cost-share. It is not difficult to verify that a budget-balanced and fair cost- 
sharing scheme for this game can be found thus: find an MST, root it at r, and 
set the cost-share for vertex f G U to be the cost of the edge from i to its parent. 
However, this scheme is not cross-monotone, and getting a cross-monotone cost- 
sharing scheme for the minimum spanning tree problem takes more work. Kent 
and Skorin-Kapov [5] and Jain and Vazirani [7] developed such budget-balanced 
and cross-monotone cost-sharing schemes for the spanning tree game using a 
directed branching game; let us denote this cost-sharing scheme by ^mst- Note 
that the values ^^mst serve also as cross-monotone, 2-approximately budget- 
balanced cost-sharing for the corresponding Steiner tree game. 

The Single Source Rent-or-Buy Game. In this paper we will consider the Single 
Source Rent-or-Buy Network Design game; this combines features of Steiner 
tree and shortest paths. The game is defined as follows: we are given a complete 
undirected graph G = (V, E) with edge costs Ce satisfying the triangle inequality, 
a special source (or root) vertex r, and a parameter M > 1. There are also a set 
of users (also called players or demands) D CV, each of which is identified with 
a vertex of the graph. We assume that there is a unique player at each vertex. 
Our results can be easily extended to multiple players, each with a weight dj 
indicating its amount of demand; we leave the details for the final version of the 
paper. 

The objective is to connect each player j to the source r via some path Pj, 
on which one unit of bandwidth has been allocated. What makes the game inter- 
esting is that aggregating paths is beneficial, in tune with the idea of economies 
of scale. Hence there are two different actions that can be performed by each 
edge: either the edge can be bought at cost Mce, but then an arbitrary amount 
of bandwidth can be sent on that edge (and hence an arbitrary number of paths 
Pj can use it); or bandwidth on the edge can be rented at cost Ce per unit band- 
width (and hence if the paths for some set S of players were using the edge, 
then the edge would cost Ce x |S'|). As usual, any edge can be used for any of 
the users. Our main theorem is the following: 

Theorem 1. There is a cross-monotone, fair cost-sharing method for the Single 
Source Rent-or-Buy network design game that is also /3 -budget-balanced, where 
j3 = 4.6. Furthermore, these cost-shares can be computed in deterministic poly- 
nomial time. 

This improves on the results of Pal and Tardos [9], who gave a 15-approximate 
cross- monotone cost-sharing scheme for the problem. They use a primal-dual 
algorithm to build the network and obtain cost-shares. 

We construct the claimed cost-shares based on expected costs in the approx- 
imation algorithm of Gupta et al. [4]. Loosely speaking, that algorithm works 
by randomly reducing the network design problem to computing Steiner trees 
over subsets of terminals; we show that if we use the function \f,MST to allocate 
approximate cost-shares to each vertex in this resulting Steiner game, then the 
expected cost share of a vertex (taken over the random reduction) gives us the 
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cross-monotone cost-shares claimed in Therorem 1. However, since the variances 
of the random variables involved may be large in general, computing these ex- 
pected costs in polynomial time is non-trivial. To this end, we give an alternate 
analysis of the algorithm of [4]; this allows us to give a derandomization using a 
small sample space, which has some additional useful properties that help ensure 
cross-monotonicity. 

Note: Independently of our result, Leonard! and Schafer (personal communica- 
tion) have also suggested deriving cross-monotone cost-shares from SimpleCFL; 
however, lacking the derandomization of the algorithm, they are not able to 
compute these cost-shares in polynomial time. 



2 The Algorithm and the Cost-Sharing 

We will use an algorithm SimpleCFL suggested by Gupta et al. [4] for the equiv- 
alent problem of connected facility location (without facility costs). First, let us 
recall the algorithm from [4]. Here a > 0 is a constant that will be chosen later. 

51. Mark each demand j independently with probability a/M, and let D' be the 
set of marked demands. 

52. Construct a minimum spanning tree T on F = D' U {r}. (This is a 2- 
approximate Steiner tree on F.) The elements of F are called the open fa- 
cilities. 

53. Assign each demand j G D to its closest demand i{j) in F. 

The algorithm suggests a simple and intuitive idea for the cost-shares: each 
player pays a cost proportional to the expected cost incurred by it on running 
the above algorithm. For a given a set of coin-tosses, the players in D' will 
pay for buying the MST, where their shares are derived from a cross-monotone 
cost-sharing scheme ^mst for the MST problem given in [5, 7]. All other players 
(players in D\D') will pay for their shortest paths to F. The cost-shares defined 
for a particular set of coin-tosses exactly cover the cost of the solution built. From 
[4] we know that this algorithm is a /3-approximation algorithm for some constant 
(3. To get a /3-approximate cost-shares we divide the above defined shares by a 
(3. Formally: 

C(Aj) = ^ F[MiMST{F,3)+t{j,F)], (2.1) 

where £{j, S) is the length of a shortest-path from j to the closest vertex of the 
set S, and the expectation is over the coin tosses. (Note that the set F = D'U{r} 
depends on the coin tosses.) 

Outline of the Proof: There are two parts to proving the result: we first have 
to show that the properties of the cost-shares claimed in Theorem 1 are in- 
deed satisfied; i.e., they are /3-budget-balanced, fair, and cross-monotone. These 
properties are proved in Section 3. 

The technically involved part of the proof involves showing that our cost- 
shares can be computed in (deterministic) polynomial time. A little thought will 
convince the reader that this is not obvious, even with the results of Gupta et 
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al. [4]. Indeed, we need to estimate the expectations in (2.1) for each of the 
players, but the random variables in (2.1) do not have small variance in general. 
Furthermore, it is unclear how to derandomize the proof of [4], since it relies on 
some severe conditioning. To take care of this problem, we give a proof of the 
performance guarantee of the SimpleCFL algorithm different from the one given 
by [4]; this is done in Section 4. Our new proof will yield a somewhat worse 
constant than that of [4], but allow us to derandomize the algorithm by marking 
the demands in Step (SI) in t-wise independent fashion for a constant t; this 
appears in Section 5. 

Using t-wise independent random choices in Step (SI), allows us to use a 
polynomial-sized sample space, letting us compute expectations such as those of 
(2.1) in (deterministic) polynomial time, by considering all the points in the sam- 
ple space. However, we need to make sure that the properties of the expectation 
proved in Section 3 (fair and cross-monotone) also hold for the expectation using 
t-wise independent random choices. Interestingly, the properties of a particular 
construction of t-wise independent random variables turns out to be crucial, as 
described in the proof of Theorem 4. 

3 Properties of the Cost-Sharing Scheme 

Recall that given an instance of the game specified by a set of demands D, a 
cost-sharing scheme for the game is simply a function ^{D,i) (with ^{D,i) = 0 
for i ^ D). We now need to show that the function defined in (2.1) is a cost- 
sharing scheme with the properties we care about, namely, approximate budget- 
balance, fairness, and cross-monotonicity. These three properties are, in fact, not 
independent. A cross-monotone and approximately budget balanced cost-sharing 
is always fair. 

Lemma 1. If a cost-sharing function ^ is approximately budget balanced and 
cross-monotone then it is also fair. 

Proof. Consider a subset AC D. By cross-monotonicity, we get ?(A^) < 

*)■ approximate budget balance property gives us *) — 

OPT(A), and completes the proof of the lemma. 

We will need the following facts about the cost-sharing scheme ^mst '■ 

Theorem 2 ([5,7]). There exists an efficiently computable cost-sharing scheme 
f.MST for the Minimum Spanning Tree game, that is budget-balanced, fair and 
cross-monotone. 

Since the spanning tree connects D and r, we use the notation ^mst{D U 
{r},f) and S,mst{D,i) interchangeably; however, we always ensure that 
f,MST{D U {r},r) = 0. To prove approximate budget-balance, we need the fol- 
lowing result bounding the performance of the algorithm SimpleCFL; its proof 
appears in Sections 4 and 5. 
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Theorem 3. The expected cost of the algorithm SimpleCFL on a set of demands 
D is at most f3 0PT{D) (where (3 = 4.6J, even if the demands are marked in a 
t-wise independent fashion in Step (SI), for a suitably large constant t. 

Note that this is a stronger form of the performance guarantee proved in [4] 
(albeit with a worse constant), since it requires only constant- wise independence 
of the random variables. Armed with this result, we are now in a position to prove 
the properties of the function f defined in (2.1), and thus to prove Theorem 1. 
The above theorems give us that f is /3-approximately budget-balanced: the 
details are deferred to the final version of the paper. 

Lemma 2. The function f is a (3 -approximately budget-balanced. 

Details of a Limited-Independence Marking: In order to prove that the ex- 
pectation using t-wise independent random marking is cross-monotone, we need 
a particular choice of random marking. Let F be a field of size > n, with |F| chosen 
large enough so that [a|F|/M]/|F| is sufficiently close to a/M; for convenience, 
we will assume that this fraction equals a/M . (This approximation impacts our 
analysis in a negligible manner, as will be shown in the full version.) Let the 
elements of the field be {oi, 02 , ..., a|F|}) and let the vertices V of the graph be 
labeled by the first n elements {oi, 02 , . . . , On}. Let S be any pre-specified sub- 
set of F with [S'! = [alFI/M]. To get a sample, generate u = {xq,x\, . . . ,Xt-i) 
uniformly at random from F‘, and define to be 1 if 

Yi = 0 otherwise. By construction, we have Pr [T) = 1] ~ a/M] furthermore, it 
is well known that the Yi generated thus are all t-wise independent. Note that 
the above distribution is generating n coin tosses Yt, while we need only |D| of 
them; however, we can just ignore all the Yj for j (/ D. 

Theorem 4. Assuming that the random marking is done using the t-wise in- 
dependent random variables explained above, or using independent random vari- 
ables, then the function £, is cross-monotone, i.e., ■C(^g) ^ 'C(^A) for any 
ACD. 

Proof. The particular type of t-wise independent distribution that we use has 
the following crucial property: for any choice of w G F*, if the set of marked 
demands in the run of SimpleCFL on A is A' {lo) and the set of marked demands 
in the run on D is D'{lo), then A' {lv) C D'{co). Define the joint probability 
p{F, E) to be the probability of selecting an element of the sample space uj such 
that the marked demands in the run of SimpleCFL on A is F’ and on D it is E. 
Note that if p{F, E) > 0, then E C E. A joint probability with this property can 
also be defined for the case when the marking is fully independent. With this 
additional property, we can prove that the scheme ^ is cross-monotone. To see 
this, note that 

aA,i) = i J2f,eP(F^E)[M fMST{F,i) + e{i,E)] 

> j EF,EP(^^f^)l^^MST(E,i)+£(i,E)]=((D,i), 

where the inequality uses the fact that the support of p{E, E) is only on pairs 
with F C E, and the cross-monotonicity of £,mst- 
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4 A New Analysis of the Performance of SimpleCFL 

Suppose first that the marking is done fully independently in Step (SI); we 
now give a way of analyzing SimpleCFL that is different from the one of [4]. We 
will then use this proof in Section 5 to show that our estimate of /3 changes 
negligible when we conduct a t-wise independent marking fro a high enough 
value of t. Recall that a solution to the Connected Facility Location Problem 
with demands D is given by the facilities F and the Steiner tree on them, with 
the cost being ^(j> *(j)) + -^c(T). Of the two terms in the sum, the former 

cost is referred to as the connection cost, and the latter is the Steiner cost. 
Let OPT be an optimal solution with facilities F* and tree T*, and let C* and 
S* = M c{T*) be the connection and Steiner costs in OPT; also let Z* = C* + S*. 

The bound for the Steiner cost C is the same as in [4] : we consider the optimal 
solution OPT, and compute the expected cost assuming that we construct the 
Steiner tree in Step (S2) using the paths of the optimal solution; the proof is 
omitted here. 

Lemma 3. The expected cost of Step (S2) o/ SimpleCFL is at most 

S = E[Mc{T)] <2{S* + aC*). (4.2) 

Massaging the Optimal Solution: Before we proceed to bound the connec- 
tion costs, let us modify OPT to get another solution OPT', which costs more 
than OPT, but which has some more structure that allows us to complete the 
argument. The new solution has the following properties: 

PI. OPT' opens facilities at F' C F*, and these are connected by a cycle T' 
instead of a tree T*; each demand j is now connected to the facility i'{j), 
which is not necessarily its closest facility. 

P2. The number of demands assigned to a facility in F' , except perhaps the root 
r, is a multiple of M. (We will call this the mod-M constraint). 

Lemma 4. There is a solution OPT' with the properties above, which has con- 
nection cost C" < C* -F S* and Steiner cost S' < 2S*. Hence, the total cost of 
OPT' isZ' <C* + 3S*. 

Proof. (Sketch) Modifying the assignment of demands to nodes in F* to satisfy 
the mod-M constraint is fairly simple. Given the tree T*, we process this in a 
two-pass fashion. The first pass goes bottom-up, starting at the leaves which 
must all be in F* , and making sure that each node satisfies the mod-M property 
by sending some b < M demand from the node to its parent in the tree. OPT 
has nodes that are not in F* , and hence cannot be facilities in F' . We eliminate 
these nodes during a top-down pass, where we distribute their demand among 
their children. 

Back to Bounding the Connection Cost: From now on, we shall only con- 
sider the modified solution OPT'. For simplicity, by making copies of nodes if nec- 
essary, we can assume that each node in F' has been assigned M demands. Recall 
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Fig. 1. The transformed instance OPT' 



that T' is a cycle which contains r. (See Figure 1.) Starting from r, let us name 
the facilities on T' in (say) clockwise order r = /p, /i, . . . , fk, fk+i = r. (Hence 
referring to fi makes sense for all I, since we can just consider I mod (fc + 1).) 

Let Di be the set of demands assigned to /;, and hence \Di \ = M for I ^ 0 hy 
assumption (P2). Let us abuse notation and add r to Dq. Let PI be the portion 
of the cycle T' joining // and //+i, and let c(P/) denote the length of Pi, and 
hence S' = M Let C'^ be the total connection cost of the demands 

in Di in the solution OPT^ (with C' = Our algorithm chooses, in 

Step (S3), the cheapest assignment of demands to nodes in F. Thus, to bound 
the expected connection cost, it suffices to bound the expected connection cost 
of an arbitrary experiment that assigns demands to nodes in F. We present one 
such experiment next and analyze it; an improved experiment will be presented 
in the final version of the paper. 

The Candidate Assignment: We now consider the following experiment: if Di H 
F ^ then we assign all the demands in Di to the element in D; n F that is 
closest to fi- Else we have to assign these demands to some other open facility 
(i.e., “go outside DC). In this case, consider the smallest t such that Fj+tHF yf 0; 
let s = l + t. Note that r G Dq, and so t < k. We now send all the demands in Di 
to the facility in Dg closest to fs ■ If we assign demands in Di to a marked node in 
Di+t, then a path for each demand in Di goes through fi, Pi, F/+i, • ■ ■ , F/+t_i, 
and then from /j+t to the element in Fj+t n F closest to it. We bound the 
expected cost of these paths, which in turn bounds the expected connection cost 
of SimpleCFL. 

Let Xi be the indicator variable that Di D F = 0 in our algorithm. (Note 
that Xq = 0 with probability 1.) Let Ai be the distance from fi and the closest 
element of DiOF. (If this is empty, Ai can be set to 0.) By the above arguments, 
the assignment cost of the M demands in Di is at most 



c[ + M EhiXi ■■■XC c{P') + M EtiiXi ■ ■ ■ X,_i) (I - XC A, (4.3) 
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Indeed, the first term is the distance traveled by the demands in Di to reach 
fi, the second term expresses the fact that demands use Pi (and pay M c{Pl)) 
if and only \i Di, ... ,Di all do not intersect F, and the third term implies that 
we incur a cost of M At if we assign demands in Di to the closest member of 
AnF. 

Note that A and Xj are independent for i ^ j, and that E[A] = (1 ~ 
a/M)^ = q. To bound M • E[(l — Xi)Ai] , let the distances from fi to the 
members of be oi < 02 < . . . < aM with aj = C'. Now, 

E[ (1 - X,)A, ] = aj X a/M x (1 - 

Note that the coefficients of Oj decrease as j increases, and hence subject to the 
constraints above, the expectation is maximized when all the aj's are equal to 

C'/M. This yields E[(l- A) A] < (C'/M) [ 1 - ( 1 -a/M)^] = (C'/M) ( 1 -q). 

Let Cl be the expected connection cost of the demands in Di in our current 
experiment. Combining the above-seen facts with the inequality (4.3), we get 
that for ^ >0, 

Cl <C[ + M Y.U c{Pi) q-'+i + Elz C-' (1 - q) q'-' ■ (4.4) 

Note that Co = Cq, adding this to the sum of (4.4) over all I, the total expected 
connection cost is 

< Eto C'l + M Eti c(E') ELi + Eli c' (i - q) EEi q*”' (4.5) 
<C' + M Eti c(P/)^ + Eli C' (1 - q) < 2 C" + 5' (4.6) 

However, q < l/e“, and hence the second term is at most l/(e“ — 1). Now 
using Lemma 4 to replace C by C* + S* and S' by 2S*, we see that the expected 
connection cost of our algorithm can be bounded by C < 2 C* -|- S* . Adding 
with (4.2), the total expected cost is at most C+S < 2{l + a)C* + {2+ S* . 
Since the optimal cost is Z* = C* + S*, we choose a « 1.35 to minimize the 
approximation ratio, and get the following result. 

Theorem 5. SimpleCFL is a (3 = 3.7 -approximation algorithm for CFL. 

We will present an improved bound of a 4.6 approximation in the final version of 
the paper; this improved algorithm can also be derandomized in a fashion similar 
to that given in Section 5. If we were to use an 1.55-approximation for Steiner tree 
[10] to buy edges in Step (S2), we would be getting an improved approximation 
ratio of 4.2 for CFL, while the analysis of [4] gets a 3.55 approximation. However, 
we need to use the 2-approximation algorithm of the MST, as there is a cross- 
monotone cost-sharing function ^mst for the MST problem. 



5 Analysis of the Limited-Independence Marking 

Let e be an arbitrary positive constant lying in (0,1). We now prove that if 
the demands are marked in t-wise independent fashion where t = alog(l/e) for 
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a suitably large constant a, then the expected approximation ratio is at most 
(1 + e) times what it is with independent marking. 

Recall that the total connection cost is a random variable that is the sum of 
three quantities: (i) the deterministic value Cj, which represents the cost of 
all demands in each Di first getting routed to fi; (ii) the value that corresponds 
to unsuccessfully traveling through some Di, and (iii) the total cost paid in 
traveling from fi to the closest marked demand in Di, once Di is identified as 
the closest cluster. We will only show that the expected total cost of (iii) gets 
multiplied by at most (1 + e) due to our t = alog(l/e)-wise independent marking; 
the argument for the term (ii) is analogous, and, in fact, simpler. Specifically, 
we will show the following. Let i be the index of some arbitrary but fixed Di. 
We show that the expected value of the random variable 

MX,X,+i ■ ■ ■ W-1 • (1 - Xi)A, 

gets multiplied by at most (1 + e). (Since i is fixed, we have not subscripted (f) 
as 4>i; this remark also holds for many other definitions below.) We next present 
some notation. 

— For each group Dj, let Zj^i, Zj^ 2 , ■ ■ ■ , Zj^m be the respective indicator random 
variables for the demands in Dj getting marked by our algorithm, when 
these demands are considered in nondecreasing order of distance from fj. 
(This ordering will be important only in the case where j = i.) 

— Let A be any set of ordered pairs {{j,k)}. Then, N{A) is the indicator 

random variable for the event “for all (j,k) G A, Zj^k = 0”. Also, 5”(A) 
denotes Zj^k- {“N” stands for none, and “S'” for sum.) 

The following lemma distills some results known from [1, 2, 11], to give two 
types of upper bounds on E [A^(A)j. 

Lemma 5. Let t\ and t 2 be any even positive integers such that ti,t 2 <t (recall 
that the marking is done in a t-wise independent manner). The following hold 
for any set A of ordered pairs {{j, fc)}. 

(i) Let LE{s,A) he the random variable denoting the inclusion- exclusion ex- 
pansion of N{A) truncated at the sth level; i.e., 

LE{s,A) = \A'\=rT\(j,k)^A' 

Then, the inequality N{A) < LE{ti,A) holds always; also, 

E [LE{D,A)] < (1 - 1/M)I^I + . (5.7) 

(That is, E[LE{ti, A)] is at most more than what it would be if 

the marking was done completely independently.) 

(ii) Let NCM(f 2 , A) denote the “normalized central moment” ^ ■ 

Then, the inequality N{A) < NCM(t 2 , A) holds always; also, 

ift 2 <\A\/M, t/ienE[iVC'M(t 2 ,A)] <8-(2t2)‘=/^-(|A|/M)-*=/2. (5.8) 
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Proof. (Sketch) The upper bounds that are claimed to hold always on N{A) 
in (i) and (ii), easily follow from the fact that ti and t 2 are even. Bound (5.7) 
follows from [2]; bound (5.8) follows from [1] (see also [11] for similar bounds). 

We now show how to use Lemma 5 to upper-bound E [<f>]. Let oi, 02 , . . . , om 
be the distances of the demands in Di from /j, written in nondecreasing order. 
Then, expanding the “(1 — Xi)Af^ part of (j), we see that 

= E]=1 MXjXj+i ■ ■ ■ W-1 • 1 

Fix k arbitrarily, and let Zk = ■ ]lfci (1 “ ' Y.]=i EE+i ' ' ' We 

aim to show that E [zk] is multiplied by at most (1 + e) in our t-wise independent 
marking, as compared to the fully-independent marking. For j = 0,1, — 1, 

define Aj = {(r, s) : (* — j) < r < (i — 1), 1 < s < M}, and Bi^k = {(*,^) : 1 < 
£ <k — 1}. Thus we get 

Zk = Z,^k-N{B,^k)-Y.]~=oN{Aj). 

Now let ai be a sufficiently large constant. Define 02 to be the smallest even 
integer that is at least 2elog(l/e), a to be ai log(l/e), ti to be the smallest even 
integer that is at least 2eai log(l/e), and £2 to be the largest even integer that 
is at most ai log(l/e)/4. Rewrite Zk as 



z,,k • N{B,,k) • E,-<a N{A,) + Z,,k • N{B,^k) ■ 

Thus, by Lemma 5, Zk is always bounded by the sum of the following two random 
variables: 



Z^^k ■ IE{a2,B,^k) ■ 

Zi^k • IE{a2, B,^k) ■ E,>a NCM{t2, Aj). 

If we expand the expectations of these two random variables using the linearity 
of expectation, we get terms each of which is a product of some of the random 
variables Z.^.; importantly, the number of factors in each such term is at most 
1 + 02 + ti and 1 -I- 02 + <2 respectively. Thus, if we choose t = 1 -I- 02 + ti (recall 
that ti > £ 2 ), then the expectations of these two random variables become 

EE,fc] •E[/E(o 2 ,B.,fc)] •E,<„E[/E(ti,^,)], and (5.9) 

E • E [/E(o 2, S,,fc)] • E [NCM{t2, A,)] (5.10) 

respectively. We next use (5.7) and (5.8) to bound these values; we will see that 
choosing the constant oi large enough results in E [z^j being at most (1 + e) 
times what it is with independent marking. The expression (5.9) is at most 

ir ^ ((1 - w)'- + (++)") ^ E,<„ (d - i,)'-” + (lf)“) 

< i d(i - + (*)") - E,<„((i - i,)“’ + (2!dallM)‘y 
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Similarly, the expression (5.10) is at most • ((1 — + (e/o 2 )“^) • 

• (2t2/i)*^^^). Thus, if the demands are marked in t-wise independent 
fashion, then E [zk] is at most-^ • ((1 — + (e/a 2 )“^) times 

Eo<,<o ((1 - 1/^)“^' + (eai log(l/e)Ai)‘0 + E,> J8 • { 21 ^ 13 ^/^). 

On the other hand, under fully-independent marking, E \zk] = ■ 

Ej>o(l — - Recalling the definitions of oi, a 2 ,ti and t 2 , it is easy to verify 
that if oi is chosen as a sufficiently large constant, then the former value is at 
most (1 + e) times the latter. 

Further Extensions: We can improve on Theorem 5 using the following ex- 
periment: for a Di with no open facilities, instead of going clockwise around the 
cycle in OPT’, we find the smallest t such that Di^t or Di-t has an open facility, 
and assign all demands in Di to this facility. This gives us a 4. 6- approximation. 
For the model of connected facility location with facility opening costs, we can 
also get constant-approximate cross-monotone cost shares by an analysis similar 
to that given here. (The proofs are deferred to the final version of the paper.) 
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Abstract. In this paper we study the approximability of the maximiza- 
tion version of constraint satisfaction problems. We provide two proba- 
bilistic approximation algorithms for Max fcCONjSAT which is the prob- 
lem to satisfy as many conjunctions, each of size at most k, as possible. 
As observed by Trevisan, this leads to approximation algorithms with 
the same approximation ratio for the more general problem Max fcCSP, 
where instead of conjunctions arbitrary fc-ary constraints are imposed. 
The first algorithm achieves an approximation ratio of The sec- 

ond algorithm achieves a slightly better approximation ratio of 
but the ratio is shown using computational evidence. These ratios should 
be compared with the previous best algorithm, due to Trevisan, that 
achieves an approximation ratio of . Both the new algorithms use 
a combination of random restrictions, a method which have been used 
in circuit complexity, and traditional semidefinite relaxation methods. 
A consequence of these algorithms is that some complexity classes de- 
scribed by probabilistical checkable proofs can be characterized as sub- 
sets of P. Our result in this paper implies that PCPc,s[log, fc] C P for any 
c/s > , and we have computational evidence that if c/s > 2*^“^ ®^ 

this inclusion still holds. 



1 Introduction 

In this paper we study the approximability of maximum /c-constraint satisfaction 
problem (Max fcCSP). An instance of Max kCSP consists of a set of Boolean 
variables and a set of weighted constraints, each acting over a fc-tuple of the 
variables. The value of an assignment is the sum of the weights of the satisfied 
constraints. This problem was defined by Khanna et al. [9] and is a natural 
generalization of many well-known optimization problems. In fact it can express 
any Max SNP problem. 

An algorithm r-approximates a maximum optimization problem if the ratio 
of the value of the solution returned by the algorithm, and the optimal value 
of the problem, is at least r. Trevisan observed in [12] that it is sufficient to 
r-approximate a restricted version of Max fcCSP in order to r-approximate 
Max fcCSP. The restricted version is called Max /cConjSAT and allows only 
constraints that are conjunctions of literals. The, to date, best approximation 
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algorithm for Max /cConjSAT, and thereby also for Max kCSP, is based on 
a linear relaxation and is due to Trevisan [12]. It achieves an approximation 
ratio of 2^“^. Since the work was first published in 1996, it has remained the 
best algorithm known and one could start to wonder if it, in fact, is the best 
possible polynomial-time approximation algorithm for Max fcCSP. In this work 
we provide two algorithms, with approximation ratios of and that 

show that this is not the case. The ratio of is shown using computational 

evidence rather than a proof. However, for the approximation ratio of 
we have a complete proof. The new approximation ratios can be considered as 
a moderate improvement, but are structurally interesting. 

In probabilistic proof checking, the problem of maximizing the accepting 
probability of a verifier looking at k bits can be described as an instance of Max 
/cConjSAT [12]. Thus, our result in this paper implies that PCPc,s[log, fc] C P 
for any c/s > and we have computational evidence that if c/s > 

this inclusion still holds. 

We point out that there exist better algorithms for Max fcCSP if the in- 
stances are satisfiable. In that case, another algorithm by Trevisan [13] achieves 
a ratio of 

In another research direction, PCP techniques have been used to show that 
approximating Max fcCSP within certain ratios, in polynomial-time, cannot 
be done unless P = NP. A long line of results led to the work of Samorod- 
nitsky and Trevisan [11], which show that there cannot exist a polynomial- 
time 2^'/*^+^“(^+^Aapproximation algorithm for Max kCSP unless P = NP. 
More recently Engebretsen and Holmerin [2] achieved an even stronger result 
showing that, for fc > 3, there cannot exist a polynomial-time 
approximation algorithm. 

Semidefinite Relaxation. Let us give a brief introduction into semidefinite re- 
laxations in connection with approximation algorithms. The work of Goemans 
and Williamson [5] laid ground for a large number of works providing better 
approximation ratios for various constraint satisfaction problems. The idea is to 
relax each clause into a set of semidefinite constraints. The resulting semidef- 
inite program can be solved to any desired accuracy in polynomial-time. The 
solution is then rounded into a solution of the original constraint satisfaction 
problem. In some cases it is possible to analyze the rounding technique used 
and thereby obtain an approximation ratio. For Max 2ConjSAT Goemans and 
Williamson [5] obtained a 0.79607-approximation algorithm. Later, Karloff and 
Zwick [8] introduced a methodology to create semidefinite relaxations for any 
constraint function and tools to analyze the rounding scheme in some cases. Us- 
ing these methods Zwick [15] obtained a 0.5-approximation algorithm for Max 
3ConjSAT and Guruswami et al. [6] obtained a 0.33-approximation algorithm 
for Max 4ConjSAT. We note that these algorithms outperform the random 
assignment algorithm with more than a factor of 3, 4 and 5, respectively. In 
contrast, the general Max fcCONjSAT by Trevisan, which is based on a linear 
relaxation instead of a semidefinite relaxation, beats the random assignment al- 
gorithm with a factor of 2. Unfortunately, applying the semidefinite relaxation 
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method on Max /cConjSAT for /c > 4 makes the analysis of the approximation 
ratio problematic. An interesting question that remains open is if the algorithm 
achieves an approximation ratio of as it does for fc < 4, but we simply 

do not know how to show it. 

Techniques Used. The main idea of both our Max fcCONjSAT algorithms is 
to make use of the fact that semidefinite relaxation methods work well for con- 
junctions of small size. We shrink the size of the conjunctions by using random 
restrictions, an often used method in circuit complexity that was introduced by 
Furst et al. [4]. The random restriction is done by assigning a random value 
to each variable with high probability. Many of the conjunctions are unsatisfi- 
able after this random restriction and are thrown away, but the effective sizes of 
the remaining conjunctions have decreased so they are tractable for traditional 
semidefinite relaxation methods. For the algorithm we use the above 

mentioned methodology of Karloff and Zwick and analyze its effect on the re- 
stricted instance. For the 2^ '^°“^ algorithm we instead use so-called gadgets [14] 
to transform the restricted instance of conjunctions into a Max 2SAT instance 
which is solved using the algorithm of Feige and Goemans [3] . 

When analyzing the approximation ratio of our 2^’®'^“^ algorithm we have 
to rely on the assumption that we have found global optima of two continuous 
optimization problems, one in ten variables and one in six variables. We note 
that this situation is quite common for many works that analyze the effective- 
ness of rounding procedures of semidefinite programs [3, 6-8, 15]^. In the case of 
our 2^ '^°“^ algorithm we rely on the Feige-Goemans algorithm which has been 
rigorously analyzed by Zwick [16]. 

Probabilistic vs. Deterministic Algorithms. We note that both our algorithms are 
probabilistic and thus the approximation ratio is calculated using the expected 
value of the produced solution. We do not see any problem derandomizing them 
with the method of Mahajan and Ramesh [10], but we have not studied it closely. 
Alternatively, multiple independent executions of the algorithm guarantees that 
except with exponentially small probability a solution with value arbitrarily close 
to the one stipulated is produced. For some applications only a lower bound of 
the optimum value is needed and we note that both algorithms can be used to 
get such a bound in a deterministic way (see Sect. 6). 

2 Preliminaries 

We say that an instance is p-satisfiable if the ratio of an optimal solution and 
the total weight of all clauses is at least p. An algorithm A a-approximates a 
maximization problem if for all instances x of the problem val(A, a;)/opt(a;) > 
a, where val(A, x) is the value of A{x) and opt(x) is the optimal value of x. 

^ Some of these results have been proved in a rigorous manner since the original 
publication and many of the assumptions are seemingly not as strong as our is, i.e. 
involve less variables. 
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Equivalently, A is said to have an approximation ratio of a. For probabilistic 
algorithms val(A, x) is allowed to be an expected value over the random choices 
done by A. 

An instance of the Max kCSP problem consists of a set {Ci, . . . ,Cm} 
of clauses (or constraints) with associated weights and a set of 

Boolean variables X = {xi, . . . , Xn}- Each clause Ci consists of a Boolean func- 
tion fi of arity k and a size k tuple of Boolean variables (xq , . . . , Xi ^ ) where 
Xi^ G X. A solution is an assignment to X and the value of the solution is the 
sum of the weights of the satisfied clauses. A clause Ci = {fi, (xq , . . . ,Xi^)) is 
satisfied if fi{xi ^^ , • ■ • , Xi,f) is true. 

An instance of the Max /cConjSAT problem is a special type of Max fcCSP 
problem where each clause is a conjunction of literals from X . A literal is either 
a variable or a negated variable. 

3 A 2^ '^°“^-Approximation Algorithm for Max /cConjSAT 

In this section we describe how our first Max fcCONjSAT algorithm works. The 
algorithm runs both a random assignment algorithm and Algorithm Al, depicted 
in Fig. 1, and returns the best solution. 



Input: A set of n Boolean variables {xi,...Xn} and a set of m conjunctions 
{Cl, . . . Cm} with weights {wi,. . . Wm}- 

1. (Initialization) Set the parameter s := 2.56. 

2. (Random restriction) Set / := 0. For each i := 1,. . . n do: 

— With probability s/k: put i in /, 7 := / U {i}. 

— With remaining probability: let Xi be assigned according to an unbiased 
coin flip. 

3. (Gadget) Set C 2 SAt := 0. For each j := 1,. . .m do: If Cj still is satishable 
given the already set variables and include at most five unassigned variables, 
apply the gadget to 2SAT as explained in Sect. 3.2. Put the 2SAT clauses in 
C 2 SAT, each with weight Wj. 

4. (Solve Max 2SAT) Apply the Feige-Goemans algorithm [3] on C 2 SAT and let 
it assign the remaining variables {xi}i£i. 



Fig. 1. Algorithm Al: a Max fcCONjSAT algorithm. 



The random assignment algorithm assigns a random Boolean value to each 
variable in the instance. The expected value of the solution is at least a 2“^- 
fraction of the total weight of all clauses. Thus, if the instance is not 2“^ '^°- 
satisfiable then the random assignment algorithm is a 2^-^°“^-approximation. 

If the instance instead is 2“^-^°-satisfiable then we can show that Algorithm 
Al produces a 2^ '^*^“^-approximation. The algorithm starts by applying a ran- 
dom restriction over the instance, thereby reducing the size of the conjunctions. 
The small conjunctions are then transformed, using gadgets, into a Max 2SAT 
instance which is solved using the algorithm of Feige and Goemans [3] . When 
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proving the approximation abilities of Algorithm Al, the main idea is to relate 
the optima of the Max fcCONjSAT instance and the produced Max 2SAT in- 
stance. Theorem 1 is a consequence of the combination of Algorithm Al and the 
random assignment algorithm. The corollary follows by using the observation 
made by Trevisan in [12]. 

Theorem 1. Max fcCONjSAT has a polynomial-time 2^ -approximation 
algorithm. 

Corollary 2. Max kCSP has a polynomial-time 2^ -approximation algo- 
rithm. 

Next we describe the two main techniques used in the algorithm, random re- 
strictions and gadgets. Then we analyze the approximation ratio of the algorithm 
and thereby prove Theorem 1. 

3.1 Random Restrictions 

In the introduction it was mentioned that semidefinite relaxation methods works 
well on small conjunctions. A Max /cConjSAT instance consists of length-fc 
conjunctions and in order to reduce the length of these conjunctions, and thereby 
making the instance tractable for semidefinite methods, we use a technique called 
random restrictions. For each variable, either assign it a random value with 
probability 1 — s/k, or let it be unassigned with probability s/k. Each clause is 
originally a conjunction of at most k variables. Restricting a clause to the values 
set by the random restriction either makes the clause true (if all literals in the 
clause were made true by the restriction), false (if any of the literals in the clause 
were made false by the restriction) or it is still a conjunction but with size i (if 
all but i literals were set and made true by the restriction) . 

3.2 Gadgets Reducing ConjSAT Clauses to 2SAT Clauses 

A gadget is a way to transform a constraint of one type into a set of constraints 
of another type. In [14] Trevisan et al. exhibit a gadget reducing a conjunction 
of three literals, Ai A A 2 A A 3 into 2SAT clauses. A 2SAT clause is a disjunction 
of literals of size at most 2. The gadget is Y, {-^Y V Ai), (^F V A 2 ), {~^Y V A 3 ), 
where F is a so-called auxiliary variable that is added. Note that if Ai A A 2 A A 3 
is true, then all four of the gadget’s clauses can be made true by setting F 
to true. However, if Ai A A 2 A A 3 is false then we can never satisfy all four 
clauses, but always three of them by setting F to false. For those familiar with 
the jargon of [14], this is a strict 4-gadget. In [14] this gadget was used to create 
a 0.367-approximation algorithm for Max SConjSAT. 

Generalizing this gadget construction we can produce a gadget reducing a 
conjunction of size i, Ai A . . . A Aj into i -\- 1 2SAT clauses: F, (^F V Ai), 

. . . (^F V Ai). (For z = 1 we do not have to use a gadget because Ai is already 
a 2SAT clause.) In Algorithm Al, after the random restriction has reduced the 
size of the conjunctions, we use the above gadget to transform conjunctions into 
2SAT clauses. The gadget is more effective for small conjunctions, and that is 
why the gadget is used only on conjunctions of size five or less. 
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3.3 Analysis of the Approximation Ratio 

If an instance is not 2“^ '^°-satisfiable, then a random assignment will be a 
2i-40-fc-approximation. Thus, left to be shown is that Algorithm A1 produce 
2i-40-fc-approximations on instances that are 2“^-40_gg^|;jgf^g^]3]^g^ 

The first phase of Algorithm A1 is to apply a random restriction, with s = 
2.56 (see Sect. 3.1). This yields an instance of fairly small conjunctions. On 
each such zConjSAT clause containing at most five literals we apply the gadget 
mentioned above, which reduces the clause to i length-2 and one length-1 2SAT 
clauses. Thus we get a Max 2SAT instance that can be solved using the Max 
2 SAT algorithm of Feige and Goemans [3] thereby deciding the variables that 
remained unassigned after the random restriction. 

The probability that an arbitrary conjunction of length k remains satisfiable 
after the random restriction and contains exactly i unassigned variables is 




Let Wtot be the sum of all weights in the Max fcCONjS AT instance. The expected 
sum of weights from length-1 clauses that an instance induce is aiWtot and the 
corresponding value for length-2 clauses is a 2 Wtot, where a± = 

02 = X)i =2 expected number of 2SAT clauses, length-1 respectively 

length-2, induced from a single conjunction of length k. 

Let the value of an optimal assignment be Wopt = P^tot, where p > 2“4-40^ 
Such an assignment, restricted to the unassigned variables, satisfy all of the 
length-2 clauses and an expected p-fraction of the weights of the length- 1 clauses 
in the Max 2SAT instance. Thus, the expected weight of satisfied length-2 
clauses is o; 2 Wtot and the expected weight of satisfied length-1 clauses is paiWtot- 

Using the Max 2SAT algorithm of Feige and Goemans [3] we are guaranteed 
to obtain a solution of expected weight at least /3ipaiWtot+/32<^2'U’tot, where /3i = 
0.976 and j32 = 0.931. The use of gadgets ensures that if we let the solution decide 
the value of the unassigned variables in the original Max /cConjSAT instance, 
the weight of that assignment will have expected value at least /SipaiWtot + 
P 2 Ct 2 Wtot — O' 2 'Wtot + PoWtot- We can now express an approximation ratio: 



PipaiWtot + P2a2Wtot - o;2Wtot + PoWtot 

pwtot 



(iiai-p ^{l-!32)a2 + p Vo • 



We see that pi (2s)*(i!)“^e“^2“^, when A: — > oo. Using s = 2.56 we get 
= 7.618 • 2“^, «2 = 27.401 • 2“^ and po = 0.0773 • 2“^ thus yielding an 
approximation ratio greater than 24-40-^ for values of k large enough. To show 
that the ratio is valid for all values of fc > 5, we have numerically checked that 
the ratio is higher than 24-40-^ for all values of k from 5 up to 1000 and then 
obtained a lower bound of 24-40-^ for all k > 1000 (details in the full paper). 
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4 A ^-Approximation Algorithm for Max /iConjSAT 

In this section we give another approximation algorithm for Max fcCONjSAT. 
This algorithm also runs two different algorithms: a linear relaxation algorithm, 
described in Sect. 4.1, and Algorithm A2, depicted in Fig. 2. The linear relax- 
ation algorithm performs well on instances that are (1 -I- 2^“^)/2-satisfiable and 
algorithm A2 achieves well on all other instances. 



Input: A set of n Boolean variables {xi,...Xn} and a set of m conjnnctions 
{Cl, . . . Cm} with weights {wi, . . . Wm}. 

1. (Initialization) Set the parameters s ~ 2.8, P := 0.72. 

2. (Semideflnite programming) Generate the semidefinite programming re- 
laxation of the inpnt instance given in Fig. 3. Find an (almost) optimal solution 
Vo, .. .Vn in polynomial-time using standard techniques. 

3. (Random restriction) Set I := 0. For each i := 1,. . .n do: 

— With probability s/k: put i in 7, 7 := 7 U {i}. 

— With remaining probability: let Xi be assigned according to an unbiased 
coin flip. 

4. (Hyperplane rounding / Random assignment) Choose a random hyper- 
plane normal uh £ S". With probability P do for each i £ I: 

— If sgn{nH ■ uo) = sgn{nH ■ Vi) assign Xi ~ 1. 

— If sgn{riH ■ no) A sgn{riH ■ Vi) assign Xi := 0. 

With remaining probability let Xi be assigned according to an unbiased coin 
flip, for all i £ I. 



Fig. 2. Algorithm A2: a Max fcCONjSAT algorithm. 



Algorithm A2 is similar in approach to Algorithm Al. The first step is to solve 
a semidefinite relaxation of the problem instance. As in the previous algorithm 
a random restriction is imposed on the variables (see Sect. 3.1), but instead of 
reducing the conjunctions of the unassigned variables into a Max 2SAT instance 
we use the solution obtained from the semidefinite relaxation to assign values to 
the remaining variables by performing a random hyperplane rounding. 

As in many previous works dealing with approximation algorithms for Max 
CSP problems using semidefinite programming we only present computational 
evidence for the approximation ratio. The result is valid if we have found the 
worst possible vector configuration in two different cases. More explicitly we rely 
on the following technical assumption. 

Assumption 3. Let P = 0.72, (vq,vi, . . .V 3 ) he a vector configuration of four 
vectors in and z he the minimum ofrelA, where A C {1, 2, 3}, defined in Fig. 
3. Then 0.493 is a lower hound on 

(P • Pr [sgn{nh ■ vq) = sgn{uh ■ vi) = . . . sgn{nh • W 3 )] + {I - P) ■ 2~^)!z , 
for all possible vector configurations of {vq, vi, . . . V 3 ). 






158 



Gustav Hast 



Furthermore, let (vo, vi, . . .V4) be a vector configuration of five vectors in R^, 
and z he the minimum of r el a, where A C {1,2, 3, 4}, defined in Fig. 3. Then 
0.337 is a lower hound on 

{P ■ Pr [sgn{nh ■ vo) = sgn{nh • ui) = . . . sgn{nh ■ ^ 4 )] + (1 - P) • , 

for all possible vector configurations of (vq, vi, . . . V4). 



max > WiZi 



Zj < relA, A C Cj,\A\ < 4 



reL = 1 



1 + Vo ■ Vi 






1 + Vo ■ Vi + Vo ■ Vj + Vi ■ Vj 

4 

. 1 “t“ Vctq ' Va,, “ 1 “ ^^^0 " ^^2 P Vcjj^ ' Va -2 



= , min 2 j 

(<T0,CTl,(T2, 0-3)6 4 

permute{{0 ,i ,j ,k}) 



= min( 






min 

(cro 

permute{{0 ,i ,j ,k ,1}) 



2 Vao ■ Vat + Jfl<t<u<4 ' 'Va^ 



Vi e S", 0 < i < 2n 

Vi+„ = —Vi, 1 <i <n 



Fig. 3 . Semidefinite program relaxation for Max fcCONjSAT. 

We have computational evidence that vector configurations of four vectors 
where all pairs of vectors have angle 1.438 are global minima and attain values 
close to the lower bound of 0.493. Vector configurations of five vectors are defined, 
up to rotation, by ten pair-wise angles. Let 9ij be the angle between Vi and Vj. 
If for some a, Oai = 1.855 for all t yf a and Oij = 1.424 for all other pair-wise 
angles, then the vector configuration seems to be a global minimum and attains 
a value close to the lower bound of 0.337. 

Due to space considerations we omit the analysis of Algorithm A2, which 
primarily consists of semidefinite relaxation techniques due to Goemans and 
Williamson [5] and Karloff and Zwick [8]. The complete analysis and proof of 
the following theorem is given in the full version of this paper. The corollary 
follows in the same way as in the previous section. 
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Theorem 4. If Assumption 3 is true, then Max fcCONjSAT has a polynomial- 
time 2^ -approximation algorithm. 

Corollary 5. If Assumption 3 is true, then Max fcCSP has a polynomial-time 
2^ -approximation algorithm. 

^From a practical standpoint Algorithm A2 seems a bit odd. First it solves 
a huge semidefinite program and after this only a small fraction of the variables 
are assigned values according to the solution. A more efficient order seems to be 
to do the random restriction first, as in Algorithm Al, and build a semidefinite 
program from the small fraction of conjunctions that still are satisfiable. This 
approach is in fact also realizable, but each weight in the semidefinite program 
has to be adjusted according to the length of its conjunction. Further details is 
given in the full version of this paper. 

Next, we give a description of the linear relaxation algorithm. After that we 
describe the semidefinite relaxation and the rounding method used in Algorithm 
A2. 

4.1 A Linear Relaxation Algorithm for Max /cConjSAT 
with Threshold Rounding 

The linear program relaxation is identical to the one by Trevisan [12]. Each 
binary variable Xi is relaxed to ti such that 0 < ti < 1. The clause Cj has an 
associated variable Zj in the relaxation. If Xi occurs positively in Cj, then Zj < ti 
is a constraint in the relaxation, and if xi occurs negated then Zj < 1 — U. It is 
easy to see that an assignment to Max fcCONjSAT can be transformed into a 
valid assignment for the linear program with the same objective value. This is 
done by letting ti = Xi. 

The difference of this algorithm compared with [12] is in how an assignment 
to the linear relaxed program is transformed into an assignment of the associated 
Max fcCONjSAT instance. In [12] the value of Xi was decided by a flip of an 
unbiased coin with probability (k — l)/k and only in the remaining case had 
the value of U some impact on the value of Xi. We instead make a threshold 
rounding by setting = 1 if U > 1/2 and Xi = 0 otherwise. 

Theorem 6. A {1 -\- e) /2-satisfiable instance o/Max fcCONjSAT, where e > 0, 
can he approximated in polynomial time within 2e/(l + e) of the optimal value. 

The key observation for proving the above theorem is that if the relaxed 
value Zj of a clause Cj is strictly larger than one half, then Cj will be satisfied 
by the threshold rounding. The proof is given in the full version of this paper. 



4.2 Semidefinite Relaxations 

Remember that a Max ConjSAT instance consists of a set of conjunctions 
{Cl, . . . , Cm} with weights (rui, . . . , Wm} and a set of variables |a;i, . . . , x„}. In 
the semidefinite relaxation each variable Xi corresponds to a vector Vi on the unit 
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sphere S'" in A variable can appear negated in a conjunction and thus 

there is for each Xi a vector Vi+n = —Vi which correspond to 'xl. This is done 
only for notational convenience and we could replace Ui+n by —Vi everywhere. 
By somewhat abusing previous notation, a conjunction Cj can be described as a 
subset of [2n] such that if Xi is part of the conjunction then i G Cj and if ^ is part 
of the conjunction then i+n G Cj. There is also a vector vq which represents true. 
Karloff and Zwick [8] introduced the so-called canonical semidefinite relaxation 
which specifies how to create the strongest possible relaxations from a specific 
constraint function. This construction could be applied on conjunctions of k 
variables, but we only use the canonical semidefinite relaxation of each subset 
of literals of size at most four. In Fig. 3 the complete semidefinite program that 
we use is specified. 

Hyperplane rounding is a popular method of obtaining a solution of the 
original problem from the solution of the semidefinite relaxation and is done as 
follows. Choose a random hyperplane in Set a;^ = 1 if Vi lies on the same 

side of the hyperplane as vq does, and otherwise set Xi = 0. We employ the 
hyperplane rounding as well, but first after making a random restriction. Thus, 
only the variables that remain unassigned after the random restriction get their 
value set by the hyperplane rounding method, all others are set by unbiased coin 
flips. 

5 Approximating Max /cConjSAT for Small Values on k 

In both Algorithm A1 and Algorithm A2 the random restriction parameter s 
was the same for all values of k. For specific values of k we can obtain better 
approximation ratios by using a larger value on s. This is the case for both 
algorithms. In Table 1 the obtained approximation ratios for Max kCSP with 
the corresponding value on s are listed for 5 < k < 12. For the approximation 
ratios of Algorithm A2, the parameter P = 0.72 was used. We note that for k < 4 
there are known semidefinite relaxation algorithms achieving better ratios. 

6 Relations with Probabilistical Proof Checking 

We use the standard notation and definitions in connection with probabilistical 
checkable proofs, PCPs. For those unfamiliar with these, we refer to the work of 

Table 1. Approximation ratios oai of Algorithm A1 (combined with a random as- 
signment) and approximation ratios q;a 2 of Algorithm A2 (combined with a linear 
relaxation with threshold rounding) on Max fcCSP for 5 < fc < 12, with tailor-made 
values on s. 
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Bellare et al. [1] which also contains the history of PCPs. The complexity class 
PCPc,s[log, g] contains all languages that have a verifier with completeness c, 
soundness s, which uses only a logarithmic number of random bits and asks at 
most q (adaptive) questions. 

The following theorem due to Trevisan shows that an approximation algo- 
rithm for Max /cConjSAT implies that certain PCP classes are contained in P. 

Theorem 7 (Trevisan [12]). If Max fcCONjSAT is r-approximable for some 
r <1, then PCPc,s[log, /c] C P for any c/s > 1/r. 

Both algorithms presented in this work are probabilistic, thus it is not imme- 
diate that the above theorem can be applied. By inspecting the proof of Theorem 
7, however, it is clear that only a lower bound of the value of an optimal solu- 
tion is needed. For the algorithm described in Sect. 4 such a lower bound can 
be obtained deterministically from the solution of the semidefinite program in 
Algorithm A2 and by performing the (deterministic) linear relaxation algorithm 
with threshold rounding. Thus, we get the following theorem. 

Theorem 8. If Assumption 3 is true, then PCPc,s[log, A:] C P for any c/ s > 

2^;— 1.54 

We now turn to the algorithm from Sect. 3.2 and primarily Algorithm Al. 
As with Algorithm A2 we do not need to perform the rounding step (in the 
Feige-Goeman algorithm), because solving the semidefinite program is enough 
to get a lower bound of the value of an optimal solution. But the random restric- 
tion that is applied before the semidefinite relaxation has to be derandomized. 
Therefore, we construct a polynomial size family of fc-wise independent random 
restrictions. As each conjunction of size k is analyzed separately we get the same 
approximation ratio if the random restriction is chosen from a family of /c-wise 
independent random restrictions. By applying Algorithm Al using each one of 
these random restrictions, it is guaranteed that at least one performs as well as 
the expected value over all different possible random restrictions. Thus, a lower 
bound on the value of the optimal solution, where the bound is at most a factor 
2^-1. 40 gjnaller than the optimal solution value is achieved. 

Theorem 9. PCPc,s[log, /c] C P for any c/s > . 
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Abstract. We study the problem of minimizing the broadcast time for a set of 
processors in a cluster, where processor pi has transmission time ti, which is the 
time taken to send a message to any other processor in the cluster. Previously, 
it was shown that the Eastest Node First method (FNF) gives a 1.5 approximate 
solution. In this paper we show that there is a polynomial time approximation 
scheme for the problems of broadcasting and multicasting in such a heterogenous 
cluster. 



1 Introduction 

Networks of Workstations (NOWs) are an extremely popular alternative to massively 
parallel machines and are widely used (for example the Condor project at Wisconsin 
[17]) and the Berkeley NOW project [16]. By simply using off-the-shelf PC’s, a very 
powerful workstation cluster can be created, and this can provide a high amount of par- 
allelism at relatively low cost. Since NOWs are put together over time, the machines 
tend to have different capabilities and this leads to a heterogenous collection of ma- 
chines, rather than a homogenous collection, in which all the machines have identical 
capabilities. 

One fundamental operation that is used in such clusters, is that of broadcast (this is a 
primitive in many message passing systems such as MPI [1,6, 8]). In addition it is used 
as a primitive in many parallel algorithms. The main objective of a broadcast operation 
is to quickly distribute the input data to the entire network for processing. Another sit- 
uation is when the system is performing a parallel search, then the successful processor 
needs to inform all other processors that the search has concluded successfully. Vari- 
ous models for heterogenous environments have been proposed in the literature. One 
general model is the one proposed by Bar-Noy et al [3] where the communication costs 
between links are not uniform. In addition, the sender may engage in another commu- 
nication before the current one is complete. An approximation factor with a guarantee 
of 0(log k) is given for the operation of performing a multicast. Other popular models 
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in the theory literature generally assume an underlying communication graph, with the 
property that only adjacent nodes in this graph may communicate. 

Broadcasting efficiently is an essential operation and many works are devoted to 
this (see [18, 9, 10, 4, 5] and references therein). In addition, for emergency notification 
an understanding of how to perform broadcast quickly is essential. 

A simple model and algorithm was proposed by Banikazemi et al [2]. In this model, 
heterogeneity among processors is modeled by a non-uniform speed of the sending 
processor. A heterogenous cluster is defined as a collection of processors pi,p 2 , ■ ■ ■ ,Pn 
in which each processor is capable of communicating with any other processor. Each 
processor has a transmission time which is the time required to send a message to 
any other processor in the cluster. Thus the time required for the communication is 
a function of only the sender. Each processor may send messages to other processors in 
order, and each processor may be receiving only one message at a time. 

Thus a broadcast operation is implemented as a broadcast tree. Each node in the tree 
represents a processor of the cluster. The root of the tree is the source of the original 
message. The children of a node pi are the processors that receive the message from pi. 
The completion time of a node is the time at which it completes receiving the message 
from its parent. The completion time of the children of pi is c, + j ■ ti, where Ci is the 
completion time of pi, ti is the transmission time of pi and j is the child number. In 
other words, the first child of pi has a completion time of ct + ti (j = 1), the second 
child has a completion time of Ci + 2ti (j = 2) etc. See Figure 1 for an example. 





(b) Optimal Solution 



Fig. 1. An example that FNF does not produce an optimal solution. Transmission times of pro- 
cessors are inside the circles. Times at which nodes receive a message are also shown. 



A commonly used method to find a broadcast tree is referred to as the “Fastest 
Node First” (FNF) technique [2]. This works as follows: In each iteration, the algo- 
rithm chooses a sender from the set of processors that have received the message (set 
S) and a receiver from the set of processors that have not yet received the message (set 
R). The algorithm then picks the sender from s G S' so that s can finish the transmission 
as early as possible, and chooses the receiver r G i? as the processor with the minimum 
transmission time in R. Then r is moved from i? to S and the algorithm continues. The 
intuition is that sending the message to fast processors first is a more effective way to 
propagate the message quickly. This technique is very effective and easy to implement. 
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In practice it works extremely well (using simulations) and in fact frequently finds op- 
timal solutions as well [2] . However, there are situations when this method also fails to 
find an optimal solution. A simple example is shown in Figure 1 . 

Despite several non-trivial advances in an understanding of the fastest node first 
method hy Liu [13] (see also work hy Liu and Sheng [15] in SPAA 2000) it was not 
well understood as to how this algorithm performs in the worst case. For example, can 
we show that in all instances the FNF heuristic will find solutions close to optimal? 

Liu [13] (see also [15]) shows that if there are only two classes of processors, then 
FNF produces an optimal solution. In addition, if the transmission time of every slower 
processor is a multiple of the transmission time of every faster processor, then again the 
FNF heuristic produces an optimal solution. So for example, if the transmission time of 
the fastest processor is 1 and the transmission time of all other processors are powers 
of 2, then the algorithm produces an optimal solution. It immediately follows that by 
rounding all transmission times to powers of 2 we can obtain a solution using FNF 
whose cost is at most twice the cost of an optimal solution'. However, this still did not 
explain the fact that this heuristic does much much better in practice. Recently, Khuller 
and Kim [12] showed that the FNF heuristic actually produces an optimal solution for 
the problem of minimizing the sum of completion times. This property is used to show 
that the FNF method has a performance ratio of at most 1.5 when compared to the 
optimal solution for minimizing broadcast time. In addition the performance ratio of 
FNF is at least || . As a corollary of the above approximation result, it is shown that if 
the transmission times of the fastest ^ processors are in the range [1 . . . C] then FNF 
produces a solution with makespan at most Tq pt + C. It is also shown that the problem 
is AP-hard, so unless P = NP there is no polynomial time algorithm for finding an 
optimal solution. 

It was conjectured in [12] that there is a polynomial time approximation scheme 
(PTAS) for this problem. In this paper we prove this conjecture. However, this algorithm 
is not practical due to its high running time, albeit polynomial. 



2 Problem Definition 

We are given a set of processors {pi,p 2 , ■ ■ - Pn) and there is one message to be broad- 
cast to all the processors. Each processor pi can send a message to another processor 
with transmission time ti once it has received the message. Each processor can be either 
sending a message or receiving a message at any point of time. Without loss of gener- 
ality, we assume that t\ < t 2 < ■ ■ ■ < tn and = 1. Also we assume that pi has the 
message at time zero. 

We define the completion time of processor pi to be the time when pi has received 
the message. Our objective is to find a schedule that minimizes Cmax = max^ Ci where 
Ci is the completion time of processor pi. In other words, we want to find a schedule 
that minimizes the time required to send the message to all the processors. 

We recall the following definition from [12]. 

' One approach to obtain a PTAS might be rounding to powers of ( 1 -|- e) . However, this does not 
work since their proof works only when the completion times of all processors are integers. 
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Definition 1. Wfe define the number o/ fractional blocks as the number of (fractional) 
messages a processor can send by the given time. In other words, given a time T, the 
number of fractional blocks of processor pi is (T — Ci)/ti. 

Our proof makes use of the following results from [12], 

Theorem 1. [12] The Fastest Node First algorithm maximizes the total number of frac- 
tional blocks for any value T. 



3 Approximation Scheme 

We now describe a polynomial time approximation scheme for the problem of perform- 
ing broadcast in the minimum possible time. Unfortunately, the algorithm has a very 
high running time when compared to the, fastest node first heuristic. 

We will assume that we know the broadcast time T of the optimal solution. Since 
t\ = 1, we know that the minimum broadcast time T is between 1 and n, and we can try 
all possible values of the form (1 -|- e)^ for some fixed e > 0 and j = 1 . 
In this guessing process we lose a factor of (1 -|- e). 

Let e' > 0 be a fixed consfanf. We define a set of fast processors F as all processors 
whose transmission time is at most e'T. Formally, F = {pj\tj < e'T}. Let S be the 
set of remaining (slow) processors. We partition S into collections of processors of 
similar transmissions speeds. For i = 1 . . .k, define Si = {pj\e'T(l + < tj < 

e'T(l + e')*} where k is [ • 

We first send messages to processors in F using FNF. We prove that there is a 
schedule with broadcast time at most (1 -|- 0(e))T such that all processors in F receive 
the message first. We then find a schedule for slow processors based on a dynamic 
programming approach. 

Schedule for F‘. We use the FNF heuristic for the set F to generate a partial broad- 
cast schedule. Assume that the schedule for F has a broadcast time of Tfmf- In this 
schedule every processor pj G F becomes idle at some time between Tfnf — tj and 
Tfnf- 

We will prove that there is a schedule with broadcast time at most (1 -|- 0(e))T 
such that all processors in F receive the message first, and then send it to the slow 
processors. The following lemma relates Tfnf with the time taken by the optimal 
schedule to propagate the message to any |U| processors. 

Lemma 1. In any schedule, we need at least Tfnf — 2e'T time units to have |U| 
processors receive (any portion of) the message. 

Proof. We prove this by contradiction. In any schedule let Ct be the number of pro- 
cessors that have completely received the message by time t. In addition, let It be the 
number of processors that have started receiving the message by time t. Suppose that at 
time T' < Tfnf — 2e'T, we have Ct' + It' > I^I- First note that It < Ct since each 
processor in It is getting the message from exactly one (distinct) processor in Ct. This 
means we should have that Ct' > \F\/2- 
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If a schedule is able to complete sending the message to at least \F\/2 processors 
by time T' , then the number of fractional blocks of this schedule is at least |F|/2. Since 
FNF maximizes fractional blocks by Theorem 1, we claim that FNF also has at least 
1^1/2 fractional blocks by time T' . Let tf be the transmission of slowest processor in 
F. Notice that in additional time f/(< e'T) all processors that had started receiving 
the message must have finished receiving it. In additional time f/(< e'T), FNF most 
certainly can double the number of processors that have received the message. Thus be- 
fore Tfmf, more than |F| processors would have received the message; a contradiction 
since Tfnf is the earliest time at which the fastest |F| processors receive the message 
in FNF. 

Lemma 2. There is a schedule in which all processors in F receive the message no 
later than any processor in S and the makespan of the schedule is at most (1 + 2>e')T. 

Proof. The main idea behind the proof is to show that an optimal schedule can be 
modified to have a certain form. Consider the set of processors of an optimal schedule 
that have received any portion of the message by time Tfnf — 2e'T. This consists of 
some fast processors, F' and some slow processors S'. Let F" = F \ F' and S" = 
S \ S'. Note that |F"| > |S"| since |F'| + |S"| < \F\ (Lemma 1). In the FNF schedule, 
by time Tfnf all processors in F’ = F' F" have the message. We can now have 
the processors in F" send messages to all processors in S' . Since |F"| > |S"| each 
processor sends only one message and this will take additional time at most e'T. By 
time Tfnf + e'T, all processors in F' U S' certainly have the message in addition 
to the processors in F" . Notice that the optimal schedule now broadcasts the message 
to all remaining processors in additional time T — {Tfnf — 2e'T). Thus we can also 
finish the broadcasting in additional time T — {Tfnf — 2e'T). The broadcast time of 
this schedule is at most Tfnf + e'T + T — {Tfnf — 2e'T) = (1 + 3e')T. 

Create All Possible Trees of S: For the processors in S, we will produce a set S of 
labeled trees T. A tree T is any possible tree with broadcast time at most T consisting 
of a subset of processors in S. Then we label a node in the tree as i if the corresponding 
processor belongs to Si (i = 1 . . . k). We prove that the number of different trees is 
constant. 

Lemma 3. The size ofS is constant for fixed e' > 0. 

Proof. First consider the size of a tree T (that is, the number of processors in the tree). 
Let us denote it as |T|. Since the transmission time of processors in S is greater than 
e'T, we need at least e'T time units to double the number of processors that received 
the message. It means that given a processor as a root of the tree, within time T we can 
have at most 2^/^ processors receive the message. Therfore, |T| < 2^/^ . Now each 
node in the tree can have different label i = 1 ... fc. To obtain an upperbound of the 
number of different trees, given a tree T we transform it to a complete binomial tree of 
size 2^/" by adding nodes labeled as 0. Then the number of different trees is at most 
{k+l)^"'^' . 

Attach T to F: Let the completion time of every processor pj G F be Cj. Each pro- 
cessor pj in F sends a message to a processor in S every tj time unit. Therefore, a 
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fast processor pj can send messages to at most Xj = [ ^ ^ J other processors. Let 
X = 6F ^j - Let us consider the time Xi of each sending point in X. We sort those 
Xi in nondecreasing order and attach a tree from S to each point (See Figure 2). Note 
that we can attach at most |2f | trees of slow processors. Clearly |2f | < n. 

We check if an attachment is feasible, using dynamic programming. Recall that we 
partition slow processors into a collection of processors Si, S 2 , ■ ■ ■ Sk{k = 

Let Si denote the number of processors in set Si. We define a state s[j, rii, rz 2 , . . . Uk] 
(0 < j < 0 < Hi < Si) to be true if there is a set of j trees in S that we can 

attach to first j sending points and the corresponding schedule satifies the following 
two conditions: i) the schedule completes by time T and ii) exactly rii processors in 
Si appear in j trees in total. Our goal is to find out whether s[j, si, S 2 , . . . Sfc] is true 
for some j, which means that there is a feasible schedule with makespan at most T. 
The number of states is at most since we need at most n trees (| Jf | < n) and 

Si < n. 

Now we prove that each state can be computed in constant time. Given s[j — 1, . . .], 
we compute s[j, rii, ri 2 , . . . Uk] as follows. We try to attach all possible trees in S to Xj. 
Then s[j, rii, ri 2 , . . . Uk] is true if there exists a tree T' such that the makespan of T' is 
at most T — Xj and s[j — l,ni — mi, ri 2 — mi , . . .Uk — nik] is true where T' has rtii 
slow processors belonging to set Si. It can be checked in constant time since the size of 
S is constant (Lemma 3). 

Theorem 2. Given a value T, if a broadcast tree with broadcast time T exists, then the 
above algorithm will find a broadcast tree with broadcast time almost (l+e')(l+3e^)T. 




Fig. 2. Attach trees for slow processors to fast processors. 
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Proof. Consider the best schedule among all schedules in which processors in F receive 
the message first. By Lemma 2, the broadcast time of this schedule is at most (l + 3e')T. 
We round up the transmission time of pj in Si to e'T(l + e')* where i is the smallest 
integer such that tj < e'(l + e')*T. By this rounding, we increase the broadcast time 
by factor of at most 1 + e'. Therefore, the broadcast time of our schedule is at most 

(l+e')(l + 3e')r. 



Theorem 3 . The algorithm takes as input the transmission times of the n processors, 
and constants e,e' > 0. The algorithm finds a broadcast tree with broadcast time at 
most (1 + e)(l + e')(l + 3e')T in polynomial time. 



Proof. We try the above algorithm for all possible value of the form T = (1 + e)-’ for 
j = 1 . . . r 1 ■ This will increase the broadcast time by factor of at most 1 + e. 

Therefore the broadcast time of our schedule is at most (1 + e)(l + e')(l + 3e')T. 

For each given value (1 + ef , we find FNF schedule for processors in F (it takes at 
most 0{n log n)) and attach trees of slow processors to processors in F, using dynamic 
programming. As we discussed earlier, the number of states is and each state 

can be checked if it is feasible in 0{{k + ) time, which is constant. Thus the 

running time of our algorithm is 0( [ jQg° 2+£) 1 n + {k + lY ^ - where 



k is [ 



log(l/e') 

log(l+e') 



1 - 



4 Multicast 

A multicast operation involves only a subset of processors. By utilizing fast processors 
which are not in the multicast group, we can reduce the multicasting time significantly. 
For example, suppose that we have m processors with transmission time t\ and m more 
processors with transmission time t2 where t\ < t2. Let we want to multicast a message 
to all processors with transmission time t 2 . If we only use processors in the multicast 
group, it will take t2 ■ log m time. But if we utilize processors with transmission time ti, 
we can finish the multicast in t\ ■ (logm + 1). Therefore, when ti t2, the speed-up 
is significant. 

Theorem 4 . We have a polynomial time approximation scheme for multicasting. 

Proof. Note that if an optimal solution utilizes k processors not in the multicast group, 
then those processors are the k fastest ones. Therefore, if we know how many processors 
participate in multicasting, we can use our PTAS for broadcasting. By trying all possible 
k and taking the best one, we have PTAS for multicasting. 
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Abstract. We consider centralized deterministic broadcasting in radio 
networks. The aim is to design a polynomial algorithm, which, given a 
graph G, produces a fast broadcasting scheme in the radio network rep- 
resented by G. The problem of hnding an optimal broadcasting scheme 
for a given graph is NP-hard, hence we can only hope for a good approx- 
imation algorithm. We give a deterministic polynomial algorithm which 
produces a broadcasting scheme working in time 0{D\ogn + log^n), 
for every n-node graph of diameter D. It has been proved recently [15, 
16] that a better order of magnitude of broadcasting time is impossible 
unless NP C In terms of approximation ratio, 

we have a Cl(log(n/D))-approximation algorithm for the radio broadcast 
problem, whenever D = l?(logn). 



1 Introduction 

A radio network is a collection of stations, equipped with capabilities of transmit- 
ting and receiving messages. Stations will be referred to as nodes of the network. 
The network is modeled as an undirected connected graph on the set of these 
nodes. An edge e between two nodes means that the transmitter of one end of e 
can reach the other end. Nodes send messages in synchronous steps (time slots). 
In every step every node acts either as a transmitter or as a receiver. A node 
acting as a transmitter sends a message which can potentially reach all of its 
neighbors. A node acting as a receiver in a given step gets a message, if and 
only if, exactly one of its neighbors transmits in this step. The message received 
in this case is the one that was transmitted by the unique neighbor. If at least 
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two neighbors v and v' of u transmit simultaneously in a given step, none of 
the messages is received by u in this step. In this case we say that a collision 
occurred at u. It is assumed that the effect at node u of more than one of its 
neighbors transmitting is the same as that of no neighbor transmitting, i.e., a 
node cannot distinguish a collision from silence. 

Broadcasting is one of the fundamental primitives in network communication. 
Its goal is to transmit a message from one node of the network, called the source, 
to all other nodes. Remote nodes get the source message via intermediate nodes, 
along paths in the network. A broadcasting scheme for a given network prescribes 
in which step which nodes transmit. In this paper we concentrate on one of the 
most important and widely studied performance parameters of a broadcasting 
scheme, which is its broadcasting time, i.e., the number of steps it uses to inform 
all the nodes of the network. Broadcasting time is considered as a function of two 
parameters of the network: the number n of nodes, and the radius D, which is 
the largest distance from the source to any node of the network. (For undirected 
graphs, the diameter is of the order of the radius.) 

In this paper we study centralized broadcasting. All nodes of the network know 
its topology, i.e., they have a labeled copy of the underlying graph as input. 
Each node also knows its label. Labels of all nodes are distinct integers. For 
simplicity, we assume that these are consecutive natural numbers with source 0 
but this assumption is not essential. Thus, the scheme itself, although executed 
in a distributed way by nodes of the network, can be considered as designed 
centrally, using the same sequential algorithm at each node. The input of this 
algorithm is the graph representing the network, and the output is a sequence 
of sets of nodes which are to transmit in consecutive steps. It is important to 
distinguish between the running time of the algorithm producing a broadcasting 
scheme for each graph and the broadcasting time of the output scheme, for a 
given graph. 

Our goal is to design a deterministic algorithm, running in polynomial time, 
which produces a fast broadcasting scheme for any input graph. The problem 
of finding an optimal deterministic broadcasting scheme for any input graph 
(i.e. a scheme having the smallest possible broadcasting time for this graph) is 
NP-hard. Hence the best we can hope for is a good approximation algorithm. 

1 . 1 Related Work 

Centralized broadcasting in radio networks has been studied, e.g., in [6, 7, 18, 
15-17]. In [6] it was shown that the problem of finding an optimal deterministic 
broadcasting scheme for any input graph is NP-hard. In [7] the authors gave a de- 
terministic polynomial algorithm which produces a broadcasting scheme working 
in time 0{Dlog^{n/D)), for any n-node graph of diameter D. Hence they got a 
0(log^(n/I?))-approximation algorithm for the radio broadcast problem. In [18] 
the authors showed a method consisting in partitioning the underlying graph 
into clusters, which improves the time of broadcasting, since known broadcast- 
ing schemes can be applied in each cluster separately, and diameters of clusters 
are smaller than the diameter of the graph. Applied to the randomized scheme 
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from [3], working in expected time 0(1? log n + log^ n), the method from [18] 
produces, for any n-node graph G of diameter D, a randomized scheme with ex- 
pected broadcasting time 0{D+log^ n). Using the broadcasting scheme from [7], 
this method constructs (in polynomial time) a deterministic broadcasting scheme 
working in 0{D + log® n) steps. 

Recently, the clustering method from [18] has been improved in [17]. Applied 
to the randomized scheme from [3], this improved method produces, for any n- 
node graph G of diameter D, a randomized scheme with expected broadcasting 
time 0{D + log^ n). On the other hand, using the broadcasting scheme from [7] 
this method constructs (in polynomial time) a deterministic broadcasting scheme 
working in 0{D + log® n) steps. On the negative side, it has been proved in 
[15] that o(log n)-approximation of the radio broadcast problem is impossible, 
unless NP C BPTI M . (Here o(logn) is meant as a multiplicative 
factor.) Under the same assumption it was also proved in [16] that there exists a 
constant c such that there is no polynomial-time algorithm which produces, for 
every n-node graph G, a broadcasting scheme with broadcasting time at most 
opt{G) + clog^ n, where opt{G) denotes optimal broadcasting time for G. 

There is also another approach to the problem of radio broadcasting, where 
the knowledge of the entire network topology is not assumed, and distributed 
broadcasting algorithms relying on limited information about the network are 
sought. It is not surprising that broadcasting under such scenarios is much slower 
than in the case of centralized broadcasting. Many authors [5, 8-10, 12, 14] stud- 
ied deterministic distributed broadcasting in radio networks assuming that every 
node knows only its own label. In [8-10, 14, 20, 13] the model of directed graphs 
was used. Increasingly faster broadcasting algorithms working on arbitrary n- 
node (directed) radio networks were constructed, the currently fastest being the 
G(nlog^ H)-time algorithm from [13]. (Here D is the radius of the network, i.e, 
the longest distance from the source to any other node). On the other hand, 
in [12] a lower bound l7(nlogH) on broadcasting time was proved for directed 
n-node networks of radius D. 

Randomized broadcasting algorithms in radio networks were studied, e.g., 
in [3, 13,22]. For these algorithms, no topological knowledge of the network was 
assumed. In [3] the authors showed a randomized broadcasting algorithm running 
in expected time 0{Dlogn + log^ n). In [21] we improved this upper bound by 
presenting a broadcasting algorithm with expected time G(Hlog(n/H)-|-log^ n). 
(Shortly later, a similar result was obtained independently in [13].) 

1.2 Our Results 

Our main result is a deterministic polynomial algorithm which produces a broad- 
casting scheme working in time 0{D log n + log^ n), for any n-node graph of di- 
ameter D. The method from [17], applied to our deterministic algorithm rather 
than to the randomized algorithm from [3], gives a deterministic polynomial al- 
gorithm producing a scheme with broadcasting time 0{D + \o^ n). (Recall that 
the 0(D + log^ n)-time scheme, obtained in [17] by applying their method to 
the randomized algorithm from [3], was itself randomized.) The negative results 
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from [15, 16] show that the order of magnitude of our broadcasting time cannot 
be improved, unless NP C 

In terms of approximation ratios, we have a 0(log(n/-D))-approximation al- 
gorithm for the radio broadcast problem, whenever D = f?(logn). The best pre- 
vious approximation ratio for the radio broadcast problem was 0(log^(n/Zl)), 
in [7]. It follows from the inapproximability result in [15] that we have the 
best possible order of approximation ratio, for D = l7(logn), unless NP C 
BPTIME{'nP^^°^ ”) ) . 

1.3 Terminology and Preliminaries 

The radio network is modeled as an undirected n-node graph G = (V,E). The 
source is denoted by 0. We denote by D the radius of the graph, i.e., the largest 
distance from the source to any node of the network. Notice that, for undirected 
graphs, the diameter is of the order of the radius. Let Lj, for j = 1, . . . , D, be 
the set of nodes of distance j from the source. Lj is called the j-th layer. We 
denote by Naiy) the set of neighbors of node v in graph G. For every node v 
in G, we define ly as the index of the layer containing node v (thus, we have 
V € Li^). A simple path from node v S Lj to node w G Lj/ is called a straight 
path, if it intersects every layer in at most one node. A node w is called a straight 
predecessor of v in graph G if < ly and there is a straight path from w to v 
in graph G. Note that v is also its own straight predecessor. 

We will use the following lemma, proved in [7]. 

Lemma 1. Let H = {R, S, E) he an undirected bipartite graph. Suppose that all 
nodes in R have the source message, and no node in S has it. There exists a set 
Q C R such that if the set of nodes transmitting in a given step is Q then at 
least [S']/ In [S'] nodes in S get the message after this step. Moreover, the set Q 
can he constructed in time polynomial in the size of H . 

In [7] the authors present an algorithm, polynomial in the size of El, to 
construct the set Q for a given undirected bipartite graph H. Iterating this algo- 
rithm 2 In [S'] times - after every step we remove all nodes in S which obtained 
the message - we get a scheme to inform at least |S'|/2 nodes in S. Call this 
scheme CW. 

We will also use another scheme. Again consider an undirected bipartite 
graph H = {R, S, E), and suppose that all nodes in R have the source message, 
and no node in S has it. The simple round-robin scheme informs all nodes in S 
in time at most [S']. Call this scheme RR. 

We will use log instead of In in later definitions and calculations, where log 
denotes the binary logarithm. 

2 The Deterministic Algorithm FDB 

This section is devoted to the description of our deterministic algorithm FDB 
(Fast Deterministic Broadcasting) which produces a broadcasting scheme work- 
ing in time G(I?logn-|-log^ n), for any n-node graph of diameter D. Fix such an 
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input graph G. For simplicity of presentation we assume that n is a power of 2, in 
order to avoid rounding of logarithms. This assumption can be easily removed. 
The scheme is divided into phases, constructed recursively, each containing 3 
stages. 

In the first stage of phase 0 only the source transmits, and in remaining 
stages of phase 0 no action is performed. Hence after phase 0 all nodes in Li 
have the source message. For every phase k > 1 and layer number j, we define 
sets Sk,j,Rk,j at the end of phase k — 1. For fc = 1 we define (at the end of 
phase 0): 

— S*!,! = Uj>i ^1.1 = -^li 

— Sij = 0 and Rij = 0 for layer number j > 1; 

Suppose that we have already constructed phases i = 0, ...,k — 1, together 
with sets S'i+i,j, Ri+i,j for all layer numbers j. We show how to construct phase k 
with corresponding sets Sk+ij, Rk+i,j- 

Let Hk^j = {Rkj, Sk,j, Ekj) be the bipartite graph with sets of nodes Rk,j, 
Skj, where the set of edges Ekj is defined as follows: {u, w} G Ekj iff G Rk,j, 
w G Sk,j and there is a straight path from w to f in graph G which is one of the 
shortest straight paths from w to any node in Rk,j- Consider the execution of 
scheme CW on graph Efkj. Since Hkj need not be a subgraph of G, we call the 
execution of scheme CW on Elkj a virtual execution. During it a node v G Rk,j 
virtually transmits a message (if the scheme CW schedules a transmission) and a 
node w G Sk,j virtually receives a message from some node in Rkj (if it would 
receive it in scheme Cw). We assume that each virtually transmitted message 
is the same for all nodes in Rk,j, and that no node in Sk,j has this message 
prior to the execution of the scheme. Actual transmitting and receiving in the 
underlying graph G (as opposed to their virtual counterparts) is referred to as 
real transmitting and receiving, or simply transmitting and receiving. 

Phase k consists of three stages. First we present the generic scheduling 
scheme (GSS(A;, j)) for phase k and layer number j, which will be used in stage 1. 

Regular Substage (Lasts 21ogn Steps): 

Run scheme CW on graph Hkj to get the set S C Skj of nodes that virtually 
receive a message in this graph. Denote by S'/. ^ the set Skj \ S. 

Closing Substage (Lasts 41ogn Steps): 

If j = (A: + 1) — log n and |5'fc ^ | <4 log n then 
apply scheme RR to the subgraph of Hk,j 
induced by nodes Rk,j and S'^. ^ n Lj+i. 

Sk+i,j '■= 0 ; 

■ — ‘5'fc+lj-t-l C Sk,j, 

else 

produce the sequence of empty sets of transmitters; 

Sk+i,j '■= Sk+i,j U Si.j; 

Sk+l,j + l + l C Sk,j \ Rk,j' 
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Output: 

List of O(logn) virtual transmitter sets: concatenation of the list from the 
regular substage (obtained by applying scheme CW , and of the list from the 
closing substage (obtained by applying scheme RR or adding the list of empty 
sets) . If the execution of scheme CW is shorter than 2 log n or the execution 
of scheme RR is shorter than 41ogn, fill the remaining steps by empty sets 
of transmitters, to provide synchronous time in execution of phases. 

Let Vk denote the set of nodes that have the source message at the end of phase 
k — 1. Note that Vi = Na{0) = Li- Now we define phase k of the scheme. 

Stage 0: 

For every layer number j, initialize Sk+i,j = 0, Rk+i,j = 0 

Stage 1: 

For r := 1 to 3 do 

For all j = r mod 3 in parallel do 

run GSS(fc, j) (sets Sk+ij were defined in this procedure) 

For every step of broadcasting, consider the union of virtual transmitter sets 
obtained for this step, over all layers j. This is the set of (real) transmitters 
to be used by the scheme in this step. 

Stage 2: 

Given the set Vk and the list of transmitter sets from stage 1, let 14+1 be the 
set of nodes that have the source message at the end of this stage. Gonstruct 
sets Rk+i,j as follows: for every node w € Sk+ij, find an arbitrary node 
V € Lj such that there is a straight path from w to v, and add node v to 

Rk+ 1 , j ■ 

Figure 1 shows phases 0 and 1 of the algorithm. Part (a) depicts phase 0, 
part (b) shows virtual transmissions during phase 1, and part (c) shows new sets 
5'2 ,i, •S' 2,2 at the end of phase 1. 

Intuition. The reason of dividing Stage 1 into three phases is to avoid interfer- 
ence between transmitters from close layers. The intuitive meaning of sets Skj 
is the following: every node of Skj has the property that all its straight pre- 
decessors in layer Lj already have the source message after phase k. Sets Rkj, 
included in layer Lj, are intended to transmit the message to sets Skj- As we 
will show in Lemma 3, messages from nodes in Rkj will be received by nodes in 
Skj n Lj+i not only virtually but also in the real execution in graph G. 

The idea of procedure GSS is to push at least half of the nodes from Skj to 
S'fc+ij+i in the regular stage, using protocol CW, and to push all these nodes 
in the closing stage, using protocol RR, if only fewer than 41ogn nodes are 
remaining. Nodes pushed out to the set S'fc+i++i, associated with the next layer 
j -I- 1, will be taken care of in phase A: -I- 1. We will prove later that if a node v 
virtually receives the message then one of its straight predecessors really receives 
the message in graph G, hence a really informed predecessor of node v is now 
in layer j -|- 1, so we put node v in the set Sfe+ij+i. We will also prove that 
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(a) 



(b) 



(c) 



nodes without source message 
nodes with source message X 

5 transmitting nodes 



nodes virtually receiving source message 
nodes with virtual collision 



Fig. 1. Phases 0 and 1 of algorithm FDB: transmissions, virtual receiving and corre- 
sponding sets Si, 1 , 52,1, 52, 2 - 



the set Sk,k+i-iogn is small, so we can, using additional sets of transmitters in 
the closing substage, push all the nodes from this set to the set Sk+i,k+2-iogn 
(not only half of them as in the regular substage). This is because, in the closing 
substage, we use protocol RR which guarantees that all nodes in set Sk,k+i-\ogn 
virtually receive a message. Since their informed predecessors are in the next 
layer now, they may be considered in the set corresponding to the next layer. 

Remark. The set Vk+i is obtained by adding to the set 14 new nodes that receive 
the source message in graph G for the first time during phase k. Although sets 
of real transmitters used in the scheme are equal to sets of virtual transmitters 
output by procedure GSS, the fact of virtual receiving of a message by node 
w G Sj, when scheme CW is executed on graph Hkj, does not necessarily mean 
that node w gets the source message in the real execution of our scheme in graph 
G in phase k. For example, if w is not a neighbor of any node in Rk,j, it does 
not get the source message, and consequently it is not added to Vk+i- 

3 Analysis of the Broadcasting Time 

In this section we show that the broadcasting scheme, produced by our algorithm 
FDB for an n-node graph G of diameter D, has broadcasting time 0{D\ogn + 
log^n). 
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Assume that n > 8 - otherwise centralized broadcasting is trivially done in 
constant time. Notice that scheme RR, working in time at most 151, completes 
broadcasting in the closing stage, if |5| < 41ogn. We will use this fact in the 
proof of Lemma 4 

Suppose that our scheme completes broadcasting in phase t. Consider phase 
k <t. 

Lemma 2. Consider sets of transmitters corresponding to the same time step 
in Stage 1 of the scheme, for a fixed r = 1,2,3. These sets are disjoint and the 
sets of their neighbors in G are also disjoint. 

Proof. This follows from the fact that, for a given r, such sets are subsets of 
layers at distance at least 3. 

Lemma 3. If node v G Sk,j, for j < ly, virtually receives a message during the 
regular substage of phase k then there is a straight predecessor of v in layer j + 1 
which receives the source message in phase k. 

Proof. The proof of the lemma follows from the following invariant: 

(a) if r: G Sk,j, for j < C, then a straight predecessor of v in layer j has the 
source message at the end of phase fc — 1; 

(b) if V G Skj, for j < ly, virtually receives a message during the regular 
substage of phase k then there is a straight predecessor of v in layer j + 1 which 
receives the source message in phase k. 

The proof is by induction on phase k. For k = 1, only set is nonempty 
among sets Sij. Consider node v G 5ip. Let w G Li be the straight predecessor 
of v. Node w has the source message at the end of phase 0 = A: — 1 by definition 
of phase 0 (in phase 0 only the source transmits and every node in Li gets 
the source message). This proves invariant (a). The arguments for part (b) are 
similar as in the induction step proved below. 

Suppose that the invariant holds for all positive integers k' < k. We prove it 
for k. First we prove invariant (a) for k. We may assume that j > 1 (otherwise, 
invariant (a) follows from the fact that all nodes in Li have the source message 
at the end of phase 0). Let w G Lj be a straight predecessor of node v. Since 
V G Sk,j, for some phase k' < k node v has been added to Sk'+ij during phase k' . 
Let w' denote a straight predecessor of v, w in layer Lj-i - it exists since j — 1 < 
j = Iw < ly and w is a straight predecessor of v. By the invariant (a) for k' we 
have that w' has the source message at the end of phase k' . By the invariant (b) 
for k' we have that w gets the source message from w' during phase k' , and 
consequently w has the source message at the end of phase k' < k — 1. 

Now we prove invariant (b) for k. Consider sets of transmitters during the 
regular substages in phase k. (Recall that, to transmit in graph G, we use the 
same sets of transmitters as in virtual transmitting.) Suppose that node v G Skj 
virtually receives a message from node w G Rkj in step I of phase k. It follows 
that w G Lj is a straight predecessor of v. Let 2 ; G Lj+i be the second node on 
the straight path from w to v. Hence node z really receives the source message in 
step 1. Indeed, its informed neighbor w (the fact that w has the source message 
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at the end of phase k—1 follows from invariant (a)) transmits in step 1. No node 
from layers j + 1 and j ’ + 2 is allowed to transmit in this step, in view of the 
division of Stage 1 into three time periods. Also, no other neighbor of z from 
Rk,j transmits, since all of them are also straight predecessors of v, and if one of 
them transmitted then v would not virtually receive the message in step 1. This 
completes the proof of the invariant and of the lemma. 

Lemma 4. For every k,j, such that j < (fc+1)— logn < n, we have = 0. 

Proof. The lemma follows from the following invariant, for every k,j, such that 
j<k + l<n + log n: 

INV(fc,j) 

(a) I [j^<J -S'fe+i.il < • ELi-iogn (z). for J > (fc + 1) - logn, 

(b) = 0, for j <{k+ 1) - logn. 

The proof of the invariant is by induction on k. It is omitted due to lack of space 
and will appear in the journal version. 

Lemma 5. If node v G does not belong to any set Sk+ij, for j = 1, . . . , 
jy — 1, at the end of phase k, then v has got the source message by the end of 
phase k. 

Proof. The proof is by induction on phase k. For k = 0, the lemma follows from 
the definition of sets Sij. Suppose that the lemma is true for phase fc — 1 > 0. 
Consider phase k. 

If node V is not in any set Skj, for j = I, . . . , — 1, then v received the 

source message by the end of phase (fc — I), in view of the inductive hypothesis. 
Suppose that v is in some set Skjy, where I < jo < jv — 1, but not in any Sk+ij, 
for j = I, . . . , j„ — I. By the description of the scheme, v cannot belong to any 
Sk+ij, for j > jo + 1. It follows that v G Sfe+ijo+i and jo + 1 = jv Two cases 
are possible: 

Case 1. jo yf (fc + 1) — logn. 

Since v G Sk+ij^+i, it follows that v has virtually received the message during 
the regular substage of phase k. Consequently, using inequality jo < jv, Lemma 3 
implies that the straight predecessor of v in layer jg + 1 = j„ has got the source 
message by the end of phase k—1. However, the only such node is v itself. This 
concludes the proof in Case 1. 

Case 2. jo = (A: + 1) — logn. 

In this case the closing substage is applied to nodes in jg during phase k. Note 
that, by Lemma 4, we have |S'fc+ijg| = 0 and \S'f. < 41ogn. Hence the natural 

properties of round-robin scheme RR imply that node v G S'j^ has got the source 
message by the end of phase k, since its predecessor does single transmission 
during scheme RR. 

Theorem 1. The broadcasting scheme, produced by the algorithm FOB for an 
n-node graph G of diameter D, has broadcasting time 0{Dlogn + log^ n). 
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Proof. Each phase of the broadcasting scheme lasts 0{log n) steps. It remains to 
show that 0{D+log n) phases are enough to complete broadcasting. By Lemma 4 
we get l = 0, for fc = D — 1 + logn and all j < D. By Lemma 5 this implies 

that all nodes get the source message by the end of phase D — 1 + log n, which 
completes the proof. 

Corollary 1. Algorithm FDB applied to the clustering method from [17] pro- 
duces a broadcasting scheme with broadcasting time 0{D + log^n), for every 
n-node graph G of diameter D. 

Corollary 2. Algorithm FDB combined with the clustering method from [17] 
yields a Oi\og{n/ D))- approximation deterministic algorithm for the radio broad- 
cast problem, whenever D = l7(logn). 

Proof. For D = 0(log'^ n), we have 0(\og{n/ D)) = C(log n), so our broadcasting 
time 0{D\ogn + log^n), exceeding the lower bound D by a factor 0(logn), 
is a 0(log(n/I?))-approximation in this range. For D = fHfog^n), we have 
0{D-\-\og^ n) = 0{D), so we have a constant approximation ratio in this range.) 

Corollary 3. The broadcasting time of the scheme produced by the algorithm 
FDB has optimal order of magnitude, unless NP C BPTIME{nP^^°^^°^'^">). 

Proof. Assume that NP C BPTIME{n^^^°^^°^'^^) does not hold. If D = 
l7(logn) then our broadcasting time is 0{D\ogn), which cannot be improved 
because the result from [15] implies that o(log n)-approximation of the radio 
broadcast problem is impossible, li D = O(logn) then our broadcasting time is 
D = 0(log^ n). A better order of magnitude would give time opt{G) + o(log^ n), 
precluded by the result from [16]. 

4 Complexity of the Algorithm FDB 

We showed that the algorithm FDB constructs a broadcasting scheme with 
broadcasting time 0{D log n + log^ n), for every n-node graph of diameter D. In 
this section we estimate the running time of the algorithm itself. 

It follows from [7] that the construction of scheme CW on an m-node graph 
can be done in time 0{m^ log to). The construction of scheme RR can be done in 
time 0{m). We now estimate the time to construct schemes used in one execution 
of procedure GSS. The regular and closing substages contain one execution of 
CW for TO = 0{n), and one execution of RR for to = O(logn), respectively. Thus 
the time to construct these schemes is 0{n^ log n). The rest of the procedure 
takes time 0{n), hence the total time of one execution of procedure GSS is 
0{n^ log n). 

Hence we get the following estimate of the running time of algorithm FDB: 
Time of stage 0 in all D phases - 0{D). 

Time of stage 1 in all D phases - 0{D) executions of GSS and finding the union 
of 0{D) disjoint sets of transmitting nodes - the total time 0{Dn^ logn). 
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Time of stage 2 in all D phases - For each node v in Sk+i,j, we find the shortest 
straight path to layer Lj. Either v G Skj and then this path is the same as 
it was, or t! G Skj-i and the new straight path is the old one without one 
node in Tj-i- The constructed sets Rkj are disjoint, hence the total time 
is 0{n). 

Theorem 2. Algorithm FOB runs in time 0{Dn^ log n) for a give n-node graph 
G of diameter D. 

Remark. We stress again the distinction between the running time of algorithm 
FDB which constructs a broadcasting scheme for a given graph G, and the 
broadeasting time of the obtained broadcasting scheme for this graph. 



5 Conclusion 

We presented a deterministic polynomial algorithm which produces a broadcast- 
ing scheme working in time 0(Dlogn + log^ n), for any n-node graph of diam- 
eter D. Combined with the clustering method from [17], our algorithm yields a 
deterministic polynomial algorithm producing a scheme with broadcasting time 
0{D + log^n). It remains open if the additive term log^n can be improved. 
More precisely: what is the fastest broadcasting scheme that can be produced 
by a polynomial deterministic algorithm, given an n-node graph of diameter D1 
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Abstract. We study the speed of convergence to approximate solutions 
in iterative competitive games. We also investigate the value of Nash 
equilibria as a measure of the cost of the lack of coordination in such 
games. Our basic model uses the underlying best response graph induced 
by the selhsh behavior of the players. In this model, we study the value 
of the social function after multiple rounds of best response behavior by 
the players. This work therefore deviates from other attempts to study 
the outcome of selfish behavior of players in non-cooperative games in 
that we dispense with the insistence upon only evaluating Nash equilib- 
ria. A detailed theoretical and practical justification for this approach is 
presented. We consider non-cooperative games with a submodular social 
utility function; in particular, we focus upon the class of valid-utility 
games introduced in [13]. Special cases include basic-utility games and 
market sharing games which we examine in depth. On the positive side 
we show that for basic-utility games we obtain extremely quick conver- 
gence. After just one round of iterative selfish behavior we are guaran- 
teed to obtain a solution with social value at least ^ that of optimal. 
For n-player valid-utility games, in general, after one round we obtain a 
^-approximate solution. For market sharing games we prove that one 
round of selfish response behavior of players gives 17( j^)-approximate 
solutions and this bound is almost tight. On the negative side we present 
an example to show that even in games in which every Nash equilibrium 
has high social value (at least half of optimal), iterative selfish behavior 
may “converge” to a set of extremely poor solutions (each being at least 
a factor n from optimal). In such games Nash equilibria may severely 
underestimate the cost of the lack of coordination in a game, and we 
discuss the implications of this. 



1 Introduction 

Traditionally, research in operation research has focused upon finding a global 
optimum. Computer scientists have also long studied the effects of lack of dif- 
ferent resources, mainly the lack of computational resources, in optimization. 
Recently, the lack of coordination inherent in many problems has become an im- 
portant issue in computer science. A natural response to this has been to analyze 
Nash equilibria in these games. Of particular interest is the price of anarchy in 
a game [8]; this is the worst case ratio between an optimal social solution and 
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a Nash equilibrium. Clearly, a low price of anarchy may indicate that a system 
has no need for a single regulatory authority. Conversely, a high price of anarchy 
is indicative of a poorly functioning system in need of some regulation. 

In this paper we move away from the use of Nash equilibria as the solution 
concept in a game. There are several reasons for this. The first reason relates 
to use of non-randomized (pure) and randomized (mixed) strategies. Often pure 
Nash equilibria may not exist, yet in many games the use of a randomized 
(mixed) strategy is unrealistic. This necessitates the need for an alternative in 
evaluating such games. 

Secondly, Nash equilibria represent stable points in a system. Therefore (even 
if pure Nash equilibria exist), they are a more acceptable solution concept if it is 
likely that the system does converge to such stable points. In particular, the use 
of Nash equilibria seems more valid in games in which Nash equilibria arise when 
players iteratively engage in selfish behavior. The time it takes for convergence to 
Nash equilibria, however, may be extremely long. So, from a practical viewpoint, 
it is important to evaluate the speed or rate of convergence. Moreover, in many 
games it is not the case that repeated selfish behavior always leads to Nash 
equilibria. In these games, it seems that another measure of the cost of the lack 
of coordination would be useful. 

As is clear, these issues are particularly important in games in which the 
use of pure strategies and repeated moves are the norm, for example, auctions. 
We remark that for most practical games these properties are the rule rather 
than the exception (this observation motivates much of the work in this paper) . 
For these games, then, it is not sufficient to just study the value of the social 
function at Nash equilibria. Instead, we must also investigate the speed of con- 
vergence (or non-convergence) to an equilibrium. Towards this goal, we will not 
restrict our attention to Nash equilibria but rather prove that after some number 
of improvements or best responses the value of the social function is within a 
factor of the optimal social value. We tackle this by modeling the behavior of 
players using the underlying best response graph on the set of strategy states. 
We consider (best response) paths in this graph and evaluate the social function 
at states along these paths. The rate of convergence to high quality solutions (or 
Nash equilibria) can then be measured by the length of the path. As mentioned, 
it may the case that there is no such convergence. In fact, in Section 4.2, it is 
shown that instead we have the possibility of “convergence” to non-Nash equi- 
libria with a bad social value. Clearly such a possibility has serious implications 
for the study of stable solutions in games. 

An overview of the paper is as follows. In section 2, we describe the problem 
formulations and model. In section 3, we discuss other work and their relation to 
this paper. In section 4, we give results for valid-utility and basic-utility games. 
We prove that in valid-utility games we obtain a ^-approximate solution if 
each player sequentially makes one best response move. For basic-utility games 
we obtain a ^-approximate solution in general, and a ^-approximate solution if 
each player initially used a null strategy. We then present a valid-utility game 
in which every Nash equilibria is at least half-optimal and, yet, iterative selfish 




Convergence Issues in Competitive Games 185 



behavior may lead to only 0(i)-approximate solutions. In section 5, we examine 
market sharing games and show that we obtain l7(j^)-approximate solutions 
after one best response move each. Finally, in section 6, we discuss other classes 
of games and present some open questions. 

2 Preliminaries 

In this section, we define necessary game theoretic notations to formally describe 
the classes of games that we study in the next sections. The game is defined as 
the tuple ([/, {Sj}, {ojO}). Here U is the set of players or agents. Associated 
with each player j is a disjoint groundset Vi, and Sj is a collection of subsets 
of Vj. The elements in the a groundset correspond to acts a player may make, 
and hence the subsets correspond to strategies. We denote player j’s strategy 
by Sj € Sj. Finally, aj : IljSj ^ K is the private payoff or utility function for 
agent j, given the set of actions of all the players. In a non-cooperative game, 
we assume that each selfish agent wishes to maximize its own payoff. 

Definition 1. A function f : 2^ ^ 'R is a set function on the groundset V. 
A set function / : 2^ ^ K is submodular if for any two sets A,B C V, we 
have f{A) + f{B) > f{A n H) -|- f{A U B). The function is non- decreasing if 
f{X) < f{Y) for any X CYC V. 

For each game we will have a social objective function 7 : UjSj K. (We 
remark that 7 can be viewed as a set function on the groundset UVi.) Our goal 
will be to analyze the social value of solutions produced the selfish behavior of 
the agents. Specifically, we will focus upon the class of games called valid-utility 
games. 

Definition 2. Let Q{U, {S'^}, {<aj}) be a non-cooperative game with social func- 
tion J. Q is a valid-utility game if it satisfies the properties: 

— J is a submodular set function. 

— The payoff of a player is at least equal to the difference in the social function 
when the player participates versus when it does not. 

— The sum of the utility or payoff functions for any set of strategies should be 
less than or equal to the social function. 

This framework encompasses a wide range of games in facility location, traffic 
routing and auctions [13]. Here, as our main application, we consider the market 
sharing game which is a special case of valid-utility games (and also congestion 
games). We define this game formally in Section 5. 

2.1 Best Response Paths 

We model the selfish behavior of players using an underlying state graph. Each 
vertex in the graph represents a strategy state S = (si, S2 , . . . , s„). The arcs in 
the graph corresponds to best response moves by the players. Formally, we have 
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Definition 3. The state graph, V = (y,S), is a directed graph. Each vertex in 
V corresponds to a strategy state. There is an arc from state S to state S' with 
label j if the only difference between S and S' is only in the strategy of player j; 
and player j plays his best response in strategy state S to go to S' . 

Observe that the state graph may contain loops. A best response path is 
a directed path in the state graph. We say that a player i plays in the best 
response path V, if at least one of the edges of V is labelled i. Assuming that 
players optimize their best response function sequentially (and not in parallel), 
we can evaluate the social value of states on a best response path in the state 
graph. In particular, given a best response path starting from an arbitrary state, 
we will be most interested in the social value of the the last state on the path. 
Notice that if we do not allow every player to make a best response on a path V 
then we may not be able to bound the social value of a state with respect to the 
optimal solution. This follows from the fact that the actions of a single player 
may be very important for producing solutions of high social value. Hence, we 
consider the following models: 

One-round path: Consider an arbitrary ordering of players ii, . . . , Path V 
is a one-round path if it starts from an arbitrary state and edges of P are 
labelled i\,i 2 , ■ ■ ■ ,in in this order. 

Covering path: A best response path 7^ is a covering path if each player plays 
at least once on the path. 

/c-Covering path: A best response path 7^ is a k-covering path if there are k 
covering paths 7^i,7^2, ■ ■ ■ ,Pk such that V = {Pi,P 2 , ■ ■ ■ ,Pk)- 

Observe that a one-round path is a covering path. Note that in the one-round 
path we let each player play his best response exactly one time, but in a cov- 
ering path we let each player play at least one time. Both of these models have 
justifications in extensive games with complete information. In these games, the 
action of each player is observed by all the other players. As stated, for a non- 
cooperative game Q with a social function 7, we are interested in the social value 
of states (especially the final state) along one-round, covering, and fc-covering 
paths. 



A Simple Example. Here, we consider covering paths in a basic load balancing 
game; Even-Dar et al. [2] considered the speed of convergence to Nash equilibria 
in these games. There are n jobs that can be scheduled on m machines. It takes 
Pj units of time for job j to run on any of the machines. The social objective 
function is the maximum makespan over all machines. The private payoff of 
a job, however, is the inverse of the makespan of the machine that the job is 
scheduled on. Thus each job wants to be scheduled on a machine with as small 
a makespan as possible. It is easy to verify that the price of anarchy in this 
game is at most 2. It is also known that this game has pure Nash equilibria 
and the length of any best-response path in this game is at most [1]. In 
addition, from any state there is a path of length at most n to some pure Nash 
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equilibrium [12]. It may, however, take much more than n steps to converge to a 
pure Nash equilibrium. Hence, our goal here is to show that the social value of 
any state at the end of a covering path is within a factor 2 of optimal. So take 
a covering path V = {S\, 82 , - ■ ■ , Sk)- Let i* be the machine with the largest 
makespan at state Sk and let the load this machine be L*. Consider the last 
job j* that was scheduled on machine i, and let the schedule after scheduling 
j* be St- Ignoring job j*, at time t the makespan of all the machines is at 
least L* — pj*. If not, job j* would not have been scheduled at machine i*. 
Consequently, we have — w(L* — Pj»). Thus, if OPT is the value of 

the optimal schedule, then opt > ~ Pj*- Clearly opt > pj» 

and so L* = L — pj* + pj» < 2opt. 



3 Related Work 

Here we give a brief overview of related work in this area. The consequences 
of selfish behavior and the question of efficient computation of Nash equilibria 
have recently drawn much attention in computer science [8, 7]. Moreover, the use 
of the price of anarchy [8] as a measure of the cost of the lack of coordination 
in a game is now widespread, with a notable success in this realm being the 
selfish routing game [11]. Roughgarden and Tardos [10] also generalize their 
results on selfish routing games to non-atomic congestion games. A basic result 
of Rosenthal [9] defines congestion games for which pure strategy Nash equilibria 
exist. Congestion games belong to the class of potential games [6] for which any 
best-response path converges to a pure Nash equilibrium. Milchtaich [5] studied 
player-specific congestion games and the length of best-response paths in this set 
of games. Even-Dar et al. [2] considered the convergence time to Nash equilibria 
in variants of a load balancing game. They bound the number of required steps 
to reach a pure Nash equilibrium in these games. Recently, Fabrikant et al. [3] 
studied the complexity of finding a pure strategy Nash equilibrium in general 
congestion games. Their PLS-completeness results show that in some congestion 
games (including network congestion games) the length of a best-response path 
in the state graph to a pure Nash equilibrium might be exponential. Goemans 
et al. [4] considered market sharing games in modeling a decentralized content 
distribution policy in ad-hoc networks. They show that the market sharing game 
is a special case of valid-utility games and congestion games. In addition, they 
give improved bounds for the price of anarchy in some special cases, and present 
an algorithm to find the pure strategy Nash equilibrium in the uniform market 
sharing game. The results of Section 5 extend their results. 

4 Basic-Utility and Valid-Utility Games 

In this section we consider valid-utility games. First we present results concerning 
the quality of states at the end of one-round paths. Then we give negative results 
concerning the non-convergence of ^-covering paths. 
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4.1 Convergence 

We use the notation from [13]. In particular, a strategy state is denoted by 
S = {si, S 2 , . . . , Sfc} G S. Here Si is the strategy of player i, where Si C Vi 
and Vi is a groundset of elements (with each element corresponding to an ac- 
tion for player i); %i corresponds to a null strategy. We also let S' 0 s' = 
{si, . . . , Si- 1 , s', Si+i, . . . , Sfc}, i.e. the strategy set obtained if agent i changes 
its strategy from Si to s'. The social value of a state S is 7 (S), where 7 is a 
submodular function on the groundset yJiVi. For simplicity, in this section we 
will assume that 7 is non-decreasing. Similar results, however, do hold in the 
general case. 

We also denote by ai{S) the private return to player i from the state S', and 
we let 7 (. (S) = 7 (SUsi) — 7 (S). Thus, formally, the second and third conditions 
in definition 2 are ai{S) > 7 (. (S 0 0i) and '^iOii{S) < 7 (S), respectively. Of 
particular interest is the subclass of valid-utility games where we always have 
oci{S) = 7 s, (S 0 0i); these games are called basic-utility games (examples of 
which include competitive facility location games). 

Theorem 1. In basic-utility games, the social value of a state at the end of a 
one-round path is at least ^ of the optimal social value. 

Proof. Let f2 = {cti, . . . , (t„} denote the optimum state, and let T = {ti, . . . , t„} 
and S = {si, . . . , s„} be the initial state and final states on the one-round path, 
respectively; we assume the agents play best response strategies in the order 
l,2,...,n. So T* = {si, . . . , Si-i,ti, . . . ,tn} is a state in our one-round path 
V = {T = T^, T^, . . . , T”+^ = S'}. Thus, by basicness and the fact that the 
players use best response strategies, we have ai(T'+^) = X)"=i ^ 

X], 7 ct, (IT* 0 0j). It follows by submodularity that > X)j 7 (r, (*5 U 

T' 0 0i) > 7 ( 0 ) — 7 (S U r) > 7 ( 12 ) — 7 (S) — 7 (r). Moreover, by basicness, 
7(S) - 7(T) = 7(T'+1) - 7(T') = E. 7(T'+1) - 7(T' 0 0.) - Ei 7(T') - 

7(T' 0 0 ^) = E.7s,(T' 0 0 .) - Eil'uiT^ ® 0 *) = Ei - E.7ar' 0 

0i). Let T* = { 01 , . . . , 0j_i, ti, . . . , tn}. Then, by submodularity, y(S) — y(T) > 
E,a,(T'+i) _ 0 0.) = - l{T)- Hence, y(^) - l{T) > 

7 ( 17 ) — 7 (S) — 27 (F). Since 7 (S) > 7 (F), it follows that 37 (S) > opt. □ 

We suspect this result is not tight and that a factor 2 guarantee is possible. 
Observe, though, that the above proof gives this guarantee for the special case 
in which the initial strategy state is F = 0. 

Theorem 2. In basic-utility games, the social value of a state at the end of a 
one-round path beginning at T = % is at least ^ of the optimal social value and 
this bound is tight. □ 

It is known that any Nash equilibria in any valid-utility game has value 
within a factor 2 of optimal. So here after just one round in a basic-utility game 
we obtain a solution which matches this guarantee. However for non-basic-utility 
games, the situation can be different. We can only obtain the following guarantee, 
which is tight to within a constant factor. 
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Theorem 3 . In general valid-utility games, the social value of some state on 
any one-round path is at least ^ of the optimal social value. 

Proof. Let j(I 2 ) = opt and assume that 7(ti, ^2, ■ ■ ■ ,tn)< Again, agent 

i changes its strategy from ti to Sj given the collection of strategies T* = 
{si, . . . , Sj_i, tj, . . . , t„}. If at any state in the path V = {T = T^,T'^, . . . = 

S} we have > ^OPT then we are done. To see this note that o;j(T*+^) > 

7(T*+^) — 7(T*00j) > 0 , since 7 is non-decreasing. Thus 7(T*+^) > 

> ai(T*+^) > ^OPT. Hence we have, 7(ti Usi, . . . , U Si, t*+i, . . .,tn)-"f{T) = 

7 ('^i ? ■ ■ ■ 7 ? ^i+i ? ■ ■ ■ 5 in) 7(^1 5 ■ ■ ■ 7 ^ j —i^i j —1 ^ i j T i j +1 ^ ■ ■ ■ 7 in) 

< E ■=! l{T^ u s,) - 7 (T^') < E ■=! 7 ( 7 ’^'+') - 7 (r^'+' © 0 ,) < E;=i 

< ^OPT. Consequently 7(0-1 U ti U si, . . . , ct* U U Si, ( 7 ^+l U ti+i, . . . , cr„ U t„) 
7(^1 C Si , . . . , U Sj , t^-i-l , . . . , tn) ^ OPT 7(^1 CSi,...,tiUS 2 , t j-i- 1 , ... , tji) 

> OPT— 7(6')— 2^ OPT > OPT. Thus, there is a j > z such that 7^^. (T*+^) > 

(fi U Si, . . . , ti U s^, U+i, ...,t„)> > ^OPT. Therefore we must 

obtain aj{T^~^^) > ^OPT for some j > i. □ 



4.2 Cyclic Equilibria 

Here we show that Theorem 3 is essentially tight, and discuss the consequences 
of this. Specifically, there is the possibility of convergence to low quality states 
in games in which every Nash equilibria is of high quality. 

Theorem 4. There are valid-utility games in which every solution on a k- 
covering path has social value at most ^ of the optimal solution. 

Proof. We consider the following n-player game. The groundset of player i con- 
sists of three elements Xi,x[ and yi. Let X = Uia;^ and X' = Uja;'. We construct 
a non-decreasing, submodular social utility function in the following manner. 
For each agent 1 < z < n, we have 



r 1 if S' n (X u X') = 0 

( 0 otherwise 



We define y, (S) in an identical manner. Finally, for each agent 1 < z < rz, we 
let yyS) = 1,VS. Clearly, the social utility function 7 is non-decreasing. To see 
that it is submodular, it suffices to consider any two sets AC B.li y^B) = 1 
then y.{A) = 1. This follows as Hn (X UX') = 0 implies that Hn (X UX') = 0. 
Hence y^{A) > 7 (,.(i?), Vz,VH C B. Similarly 7 (,/(A) > 7 (./(S), Vz,VA C B. 
Finally, y,{A) = jy^B) = 1, Vz,VA C B. It is well known that a function / is 
submodular if and only if A C B implies /j(A) > fj(B), \/j G V — B. Thus 7 is 
submodular. 

With this social utility function, we construct a valid utility system. To do 
this, we create private utility functions ai using the following rule (except for a 
few cases given below), where Xi = S {xi U x'f). 



a* (S') 



|(xujf')ns| 

IWI 

|(xux')ns| 



if yt G Sj 
if Vi i Sj 
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In the following cases, however, we ignore the rule and use the private utilities 
given in the table. 



Si 


S2 


S3 




Sn—1 


Sn 


ai{S) 


Q!2(S') 


asiS) 




Q!n-l(5') 


an{S) 


Xi 


X2 


03 




0n-l 


0„ 


0 


1 


0 




0 


0 


Xi 


X2 


X3 




0n-l 


0n 


0 


0 


1 




0 


0 


Xi 


X2 


X3 




Xn—1 


0„ 


0 


0 


0 




1 


0 


Xi 


X2 


X3 




Xn—1 


Xn 


0 


0 


0 




0 


1 


x'l 


X 2 


X3 




Xn —1 


Xn 


1 


0 


0 




0 


0 


x'l 


x '2 


X3 




Xn —1 


Xn 


0 


1 


0 




0 


0 


x'l 


X '2 


X3 




X'n-l 


Xn 


0 


0 


0 




1 


0 


x'l 


x '2 


X3 




x'n-1 


X'n 


0 


0 


0 




0 


1 


Xi 


x '2 


X'3 




x'n -1 


x' 


1 


0 


0 




0 


0 


Xi 


X 2 


X'3 




x'n -1 




0 


1 


0 




0 


0 


Xi 


X 2 


X3 




^n —1 




0 


0 


0 




1 


0 



Observe that, by construction, = l{S) for all S (including the excep- 

tions). It remains to show that the utility system is valid. It is easy to check 
that ai{S) > 7 ( 5 ') — 7(5” 0 0i) = 0 0i) for the exceptions. So consider the 

“normal” S. If Si fl (xi U x') = 0, then ai{S) = I when i/i G Si and ai{S) = 0 
otherwise. In both cases ai{S) = 7 '(S' 0 0i). If Si 0 (xi U x') yf 0 then 






2 

I 

I 

0 



if Ui G Si and {S — Si) n (X U X') = 0 

if y, G S, and [s - Si) n (X U X') yf 0 

if yi ^ Si and \s — Si) n (X U X') = 0 

if y, ^ S, and (S' - S,) n (X U X') yf 0 



Consider the first case. We have that o;'(S) = I 0 |(xux"')n 5 | “ 

^ + |(xux')nSi|+|(xux')n(S-Si)| = 1 + 1 x 7+0 = 2 = 7 '(S 0 0*). It is easy to 
verify that in the other three cases we also have Oi(S) > y'(S 0 0i). Thus 
our utility system is valid. It remains to choose which subsets of each players’ 
groundset will correspond to feasible strategies in our game. We simply allow 
only the singleton elements (and the emptyset) to be feasible strategies. That 
the set of possible actions for player i are Ai = {0+ x+ x', y^}. Now it is easy 
to see that an optimal social solution has value n. Any set of strategies of the 
form {yi, . . . ,yi_i,Zj,yi+i, . . . ,y„}, where Zi G {xi,x',yi}, I < i < n gives a 
social outcome of value n. However, consider the case of n = 3. From the start 
of the game, if the players behave greedily then we can obtain the sequence of 
strategies illustrated in Figure 1. The private payoffs given by these exceptional 
strategy sets mean that each arrow actually denotes a best response move by 
the labelled agent. However, all of the (non-trivial) strategy sets induce a social 
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Fig. 1. Bad Cycling. 



outcome of value 1, a factor 3 away from optimal. Clearly this problem general- 
izes to n agents. So we converge to a cycle of states all of whose outcomes are a 
factor n away from optimal. □ 

So our best response path may lead to a cycle on which every solution is 
extremely bad socially, despite the fact that every Nash equilibria is very good 
socially (within a factor two of optimal) . We call such a cycle in the state graph 
a cyclic equilibria. The presence of low quality cyclic equilibria is therefore dis- 
turbing: even if the price of anarchy is low we may get stuck in states of very 
poor social quality! We remark that our example is unstable in the sense that 
we may leave the cyclic equilibria if we permute the order in which players make 
there moves. We will examine in more detail the question of the stability of cyclic 
equilibria in a follow-up paper. 



5 Market Sharing Games 

In this section we consider the market sharing game. We are given a set U of n 
agents and a set H of m markets. The game is modelled by a bipartite graph 
G = {H U [/, E) where there is an edge between agent j and market i if market 
i is of interest to agent j (we write j is interested in market i). The value of 
a market i G H is represented by its query rate qi (this is the rate at which 
market i is requested per unit time). The cost, to any agent, of servicing market 
i is Ci- In addition, agent j has a total budget Bj. It follows that a set of 
markets Sj C H can be serviced by player j if G < Bj-, in this case we 

say that Sj represents a feasible action for player j. The goal of each agent is to 
maximise its return from the markets it services. Any agent j receives a reward 
(return) Ri for providing service to market i, and this reward is dependent upon 
the number of agents that service this market. More precisely, if the number of 
agents that serve market i is then the reward Rt = ^. Observe that the total 
reward received by all the players is equal to the total query rate of the markets 
being serviced (by at least one player). The resultant game is called the market 
sharing game. Observe that if aj{S) is the return to agent j from state S, then 
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the social value is 7(5) = is then easy to show that the market 

sharing game is a valid-utility game [ 4 ]. We remark that the subcase in which 
all markets have the same cost is called the uniform market sharing game] the 
subcase in which the bipartite graph G is complete is called the complete market 
sharing game. Note that in this game, the strategy of each player is to solve 
a knapsack problem. Therefore, in order to model computationally constrained 
agents, we may assume that the agents apply A-approximation algorithms to 
determine their best-response strategies. We then obtain the following theorems 
concerning the social value after one round of best responses moves. 

Theorem 5 . In market sharing games, the social value of a state at the end 
of a one-round path is at least 2 h\+i A optimal social value ( or at least 
-n A the agents use X- approximation algorithms). 

Proof. Let f 2 = {ai, . . . ,an} denote an optimum state. Here aj C H is the 
set of markets that player j services in this optimum solution; we may also 
assume that each market is provided by at most one player. Let T = {ti, ... , 
and S = {si,...,s„} be the initial state and final states on the one-round 
path, respectively. Again, we assume the agents play best response strategies 
in the order 1 , 2 , . . . , n. So in step r, using a A-approximation algorithm, agent 
r changes its strategy from G to s^; thus T'’ = {si , . . . ,Sr, G+i, . . . , tn} is an 
intermediate state in our one-round path V = {T = ,T^ , . . . ,T” = S}. Let 

aj{S) be the return to agent j, then the social value of the state S' = T" is 
7(S) = J 2 j^u So to show that ^ i+(A+i)g„ 

Towards this goal, we first show that 7(S) = > j)- 

We know that agent j does not changes its strategy from s^ after step r. Therefore 
a market i has a nonzero contribution in 7(S) if and only if market i has a nonzero 
contribution in the summation X je [/ ^3 ) ■ For any market i, Hi appears in one 

of strategies in T" then the contribution of i to 7(S) is qi. On the other hand, at 
most n players use market i in their strategies. Consequently, the contribution of 
market i in the summation Xjec/ aj{T^) is at most (I-I-5 + 5 + . . - + ^)qi = H^qi. 
It follows that > 777 required. We denote by T 

the summation Xjet/ ^3 A' A- Next consider the optimal assignment 12 , and let 
Yj be the set of markets that are in serviced by agent j in Uj but that are not 
serviced by any agent in T”, that is, Yj = aj — U^et/Sr- Now 7(6') is greater than 
the value of all the markets in — Yr) since these markets are a subset of 

markets serviced in T”. Hence, using the notation q{Q) = J 2 u£Q di to denote the 
sum of query rates of a subset Q of the markets, we have 7(6') > Xre(7 A^r — Yr). 
Next we will prove that P > \ ^reu A^r)- Let Y) be the markets in Yj that are 
serviced in P 2 , that is, Y) = Yj — (siU- • •Us^Ufj+iU- • -Utn). Then Y) is a feasible 
strategy for agent j at step j, and thus, since player j uses a A-approximation 
algorithm, we have Xaj{T 2 ) > q{Y'). Therefore, AT > Yl,r&u A^A- 

Finally, we claim that T > '^j^jj AY")- To see this, consider a any market 
i G Yj = Yj — Y). Then market i is not in the strategy set of any agent in 
T”, but is in the strategy set of at least one player in . Therefore, somewhere 
on the path V after P 2 some player must change its strategy and discontinue 
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servicing market i. Let bi be time step such that is the first state amongst 
, T” that does not service market i. Let Mj = {i ^ H\bi = j} he the set 
of markets for which bi = j. It follows that uY"{t) = U,e jjMr- Notice that 
Mr C tr and no other agents service any market in Mr at step r. It follows that 
a,{T^) > q{Mj). Therefore, E.et/ 

Hence we have, opt = ^ ^jeu ~ ^i) + Eiet/ AA) < 7(5') + 

E.err AY^) + E.etr AY”) < l{S) + AT + T < (1 + (A + 1)H„)7(5). □ 

Theorem 6. In market sharing games, the social value of a state at the end of 
a one-round path may be as bad as of the optimal social value. In particular, 
this is true even for uniform market sharing games and complete market sharing 
games. 

Proof. Consider the following instance of a complete market sharing game. There 
are m = n markets, and the query rate of market i is qi = j — e for all 1 < z < n 
where e is sufficiently small. The cost of market z is Cj = 1 + (rz — z)e for 2 < i < n 
and Cl = 1. There are n players and the budget of player j is equal to 1 + (zz— j)e. 
Consider the ordering 1,2, ... ,n and the one-round path starting from empty 
set of strategies and letting each player play once in this order. The resulting 
assignment after this one-round path is that all players provide market number 
1 and the social value of this assignment is n — e. However, the optimum solution 
is for agent j to service market j giving an optimal social value of rziL„ ~ ne. 
Thus, the ratio between the optimum and the value of the resulting assignment 
is Hn at the end of a one-round path. 

The bad instance for the uniform market sharing game is similar. There are 
n markets of cost 1 and query rates qi = j — e for all 1 < z < zz where e is 
sufficiently small. There are zz players each with budget 1. Player j is interested 
in markets j, j -I- 1, . . . , zz and in market 1. It follows that the social value of the 
assignment after one round is zz — e. The optimal social value covers all markets 
and its value is rziL„ ~ ne. Thus, the ratio is -jj- after one round. □ 

6 Conclusion and Open Problems 

In this paper, we presented a framework for studying speed of convergence to 
approximate solutions in competitive games. We proved bounds on the out- 
come of one round of best responses of players in terms of the social objective 
function. More generally, one may consider longer (but polynomial-sized) best- 
response paths, provided the problem of cycling can be dealt with. In acyclic state 
graphs, such as potential games (or congestion games), the PLS-completeness 
results of Fabrikant et al. [3] show that there are games for which the size of 
the shortest best-response path from some states to any pure Nash equilibrium 
is exponential. This implies that in some congestion games the social value of 
a state after exponentially many best responses might be far from the optimal 
social value. However, this does not preclude the possibility that good approxi- 
mate solutions are obtained when short /c-covering paths are used. This provides 
additional motivation for the study of such paths. Here we may consider using 
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a local optimization algorithm and evaluating the output of this algorithm after 
a polynomial number of local improvements. 

The market sharing games are not yet well understood. In particular, it is 
not known whether exponentially long best-response paths may exist. Bounding 
the social value of a vertex at the end of a fc-covering path is another open 
question. Goemans et al. [4] give a polynomial-time algorithm to find the pure 
Nash equilibrium in uniform market sharing games. Finding such an equilibrium 
is NP-complete for the general case, but the question of obtaining approximate 
Nash equilibria is open. 



Acknowledgments 

This work began at the Bell-Labs Algorithms Pow-Wow; we thank Bruce Shep- 
herd for hosting us at Bell-Labs. The first author would also like to thank Michel 
Goemans for useful discussions. 



References 

1. P. Brucker, J. Hurink, and F. Werner. “Improving local search heuristics for some 
scheduling problems”, Discrete Applied Mathematics, 72 , pp47-69, 1997. 

2. E. Even-dar, A. Kesselman, and Y. Mansour. “Convergence time to Nash equilib- 
ria”, I CALF, 2003. 

3. A. Fabrikant, C. Papadimitriou, and K. Talwar. “On the complexity of pure equi- 
libria”, STOC, 2004. 

4. M. Goemans, L. Li, V.S. Mirrokni, and M. Thottan. “Market sharing games applied 
to content distribution in ad-hoc networks”, MOBIHOC, 2004. 

5. I. Milchtaich. “Congestion games with player-specific payoff functions”. Games and 
Economic Behavior, 13 ( 1 ), pplll-124, 1996. 

6. D. Monderer and L. Shapley. “Potential games”. Games and Economic Behavior, 
14(1), ppl24-143, 1996. 

7. N. Nisan and A. Ronen. “Algorithmic mechanism design”. Games and Economic 
Behavior, 35 ( 1 - 2 ), ppl66-196, 2001. 

8. C. Papadimitriou. “Algorithms, games, and the internet”, STOC, 2001. 

9. R. W. Rosenthal. “A class of games possessing pure-strategy Nash equilibria”, 
International Journal of Game Theory, 2 , pp65-67, 1973. 

10. T. Roughgarden and E. Tardos. “Bounding the inefficiency of equilibria in 
nonatomic congestion games”. Games and Economic Behavior, 47 ( 2 ), pp389-403, 
2004. 

11. T. Roughgarden and E. Tardos. “How bad is selfish routing?”, J. ACM, 49 ( 2 ), 
pp236-259, 2002. 

12. P. Schuurman and T. Vredeveld. “Performance guarantees of local search for mul- 
tiprocessor scheduling”, IPCO, 2001. 

13. A. Vetta. “Nash equilibria in competitive societies, with applications to facility 
location, traffic routing and auctions”, FOGS, 2002. 




Cuts and Orderings: On Semidefinite 
Relaxations for the Linear Ordering Problem 



Alantha Newman* 

MIT CSAIL, Cambridge, MA 02139 
alanthaOtheory . csail .mit . edu 



Abstract. The linear ordering problem is easy to state: Given a com- 
plete weighted directed graph, find an ordering of the vertices that maxi- 
mizes the weight of the forward edges. Although the problem is NP-hard, 
it is easy to estimate the optimum to within a factor of 1/2. It is not 
known whether the maximum can be estimated to a better factor us- 
ing a polynomial-time algorithm. Recently it was shown [NVOl] that 
widely-studied polyhedral relaxations for this problem cannot be used to 
approximate the problem to within a factor better than 1/2. This was 
shown by demonstrating that the integrality gap of these relaxations is 
2 on random graphs with uniform edge probability p = jn. In this 

paper, we present a new semidefinite programming relaxation for the lin- 
ear ordering problem. We then show that if we choose a random graph 
with uniform edge probability p — where d = w(l), then with high 
probability the gap between our semidefinite relaxation and the integral 
optimal is at most 1.64. 



1 Introduction 

Vertex ordering problems comprise a fundamental class of combinatorial op- 
timization problems that, on the whole, is not well understood. For the past 
thirty years, combinatorial methods and linear programming techniques have 
failed to yield improved approximation guarantees for many well-studied ver- 
tex ordering problems such as the linear ordering problem and the traveling 
salesman problem. Semidefinite programming has proved to be a powerful tool 
for solving a variety of cut problems, as first exhibited for the maximum cut 
problem [GW95]. Since then, semidefinite programming has been successfully 
applied to many other problems that can be categorized as cut problems such 
as coloring fc-colorable graphs [KMS98], maximum-3-cut [GW04], maximum k- 
cut [FJ97], maximum bisection and maximum uncut [HZOl], and correlation 
clustering [GGW03], to name a few. In contrast, there is no such comparably 
general approach for approximating vertex ordering problems. 

In this paper, we focus on a well-studied and notoriously difficult combi- 
natorial optimization problem known as the linear ordering problem. Given a 
complete weighted directed graph, the goal of the linear ordering problem is to 
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find an ordering of the vertices that maximizes the weight of the forward edges. 
Although the problem is NP-hard [Kar72], it is easy to estimate the optimum 
to within a factor of In any ordering of the vertices, either the set of forward 
edges or the set of backward edges accounts for at least half of the total edge 
weight. It is not known whether the maximum can be estimated to a better 
factor using a polynomial-time algorithm. Approximating the problem to within 
better than || is NP-hard [NVOl]. 

The linear ordering problem is also known as the maximum acyclic subgraph 
problem. Given a weighted directed graph, the maximum acyclic subgraph prob- 
lem is that of finding the maximum weight subgraph that contains no cycles. 
The forward edges in any linear ordering comprise an acyclic subgraph and a 
topological sort of an acyclic subgraph yields a linear ordering of the vertices in 
which all edges in the acyclic subgraph are forward edges. 

Recently it was shown that several widely-studied polyhedral relaxations for 
the linear ordering problem each have an integrality gap of 2, showing that it is 
unlikely these relaxations can be used to approximate the problem to within a 
factor greater than ^ [NV01,New00]. The graphs used to demonstrate these inte- 
grality gaps are random graphs with uniform edge probability of approximately 
2'/*°®”/n, where n is the number of vertices. For sufficiently large n, such a ran- 
dom graph has a maximum acyclic subgraph close to half the edges with high 
probability. However, each of the polyhedral relaxations studied provide an up- 
per bound for these graphs that is asymptotically close to all the edges, which 
is off from the optimal by a factor of 2. 

In this paper, we first present a new semidefinite programming relaxation 
for the linear ordering problem. A vertex ordering for a graph G = (V, E) with 
n vertices can be fully described by a series of n -I- 1 cuts. We use this simple 
observation to relate cuts and orderings. We derive a semidefinite program for 
the linear ordering problem that is related to the semidefinite program used 
in the Goemans- Williamson algorithm to approximate the maximum cut prob- 
lem [GW95]. We note that by using different objective functions, our semidefinite 
programming relaxation can be used to obtain semidefinite relaxations for many 
other vertex ordering problems. 

Second, we show that for sufficiently large n, if we choose a random directed 
graph on n vertices with uniform edge probability P = ^ (i.e. every edge in 
the complete directed graph on n vertices is chosen with probability p), where 
d = u;(l), our semidefinite relaxation will have an integrality gap of no more 
than 1.64 with high probability. In particular, the graphs used in [NVOl] to 
demonstrate integrality gaps of 2 for the widely-studied polyhedral relaxations 
fall into this category of random graphs. The main idea is that our semidefinite 
relaxation provides a “good” bound on the value of an optimal linear ordering 
for a graph if it has no small roughly balanced bisection. With high probability, 
a random graph with uniform edge probability contains no such small balanced 
bisection. 
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2 Relating Cuts and Orderings 

Given an undirected weighted graph G = (V,E), the maximum cut (maxcut) 
problem is to find a bipartition of the vertices that maximizes the weight of the 
edges crossing the partition. In 1993, Goemans and Williamson used a semidefi- 
nite programming relaxation to obtain a .87856-approximation algorithm for this 
fundamental graph optimization problem [GW95]. The goal of the Goemans- 
Williamson algorithm for the maxcut problem is to assign each vertex i C G a 
vector Vi G {1,-1} so as to maximize the weight of the edges (i,j) such that 

Vi ^ Vj . 

A closely related graph optimization problem is the maximum directed cut 
(dicut) problem. Given a directed weighted graph G = (V) A), the dicut problem 
is to find a bipartition of the vertices - call these disjoint sets Vi and V 2 - that 
maximizes the weight of the directed edges {i,j) such that vertex i is in set Vi 
and vertex j is in set V 2 . Note that the edges in a directed cut form an acyclic 
subgraph. We can generalize the dicut problem to that of dividing the vertices 
into k labeled sets Vi,V 2 , ■ ■ ■ ,Vk so as to maximize the weight of the edges (i,j) 
such that vertex i is in set 14 and vertex j is in set Vh and k < h. We call this 
the fc-acyclic dicut problem. The linear ordering problem is equivalent to the 
n-acyclic dicut problem. 



2.1 A Relaxation for the Linear Ordering Problem 

We can generalize the semidefinite programming relaxation for the dicut prob- 
lem [FG95,GW95] to obtain a new semidefinite programming relaxation for the 
linear ordering problem. The basic idea behind this formulation is a particular 
description of a vertex ordering that uses n-l- 1 unit vectors for each vertex. Each 
vertex i gV has n -I- 1 (n = |E|) associated unit vectors: w?, . . . w”. In an 

integral solution, we enforce that vf = —1, vf = I and that wf and differ 
for only one value oi h, 0 < h < n. Gonstraint (1) enforces that in an integral 
solution, and differ for only one such value of h. This position h denotes 
vertex i’s position in the ordering. For example, suppose we have a graph G that 
has four vertices, arbitrarily labeled 1 through 4. Gonsider the vertex ordering 
in which vertex i is in position i. An integral description of this vertex ordering 
is: 



{vl 
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vl. 


vl} = 
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1, 


1, 


1, 


1}, 
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-1, 


-1,- 


-1,- 


-1, 


!}■ 



Let G = (V, A) be a directed graph. The following is an integer quadratic 
program for the linear ordering problem. For the sake of convenience, we assume 
that n is odd since this simplifies constraint (2). By P{G), we denote the optimal 
value of the integer quadratic program P on the graph G. 
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We obtain a semidefinite programming relaxation for the linear ordering problem 
by relaxing constraint (3) to: G 7?.”, We denote the optimal value of 

the relaxation of P on the graph G as Pr{G). 



2.2 Cuts and Uncuts 



Suppose we have a directed graph G = {V, A) and we are given a set of unit 
vectors {vi} G 7^", 0 < i < n. We will define the forward value of this set 
of vectors as the value obtained if we compute the value of the dicut semidefi- 
nite programming relaxation [GW95,FG95] using these vectors. Specifically, the 
forward value for this set of vectors is: 



max 




- Vt ■ Vj - Vo ■ Vi + Vo ■ Vj). 



( 4 ) 



In an integral solution for the dicut problem, there will be edges that cross the 
cut in the backward direction, i.e. they are not included in the dicut. For a 
specified set of unit vectors, we can view the dicut semidefinite programming 
relaxation as having forward and backward value. We define the backward value 
of the set of vectors {ui} as: 



max i(l - ■ Vj - Vo ■ vj + vo ■ Vi). 



^ r 

ij^A 



( 5 ) 



The difference between the forward and backward value of a set of vectors {vi} 
is: 



E ">Jo-Vi-vo)- 

ijeA 



( 6 ) 



Lemma 1. If a directed graph G = {V, A) has a maximum acyclic subgraph of 
+ ^)|^| edges, then there is no set of vectors {ui} such that the difference 
between the forward and backward value of this set of vectors exceeds 2(5|A|. 
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Proof. We will show that given a vector solution {ui} to the semidefinite program 
in which the objective function is (6) and all the Vi vectors are unit vectors, we 
can find an integral (i.e. an actual cut) solution in which the difference of the 
forward and backward edges crossing the cut is exactly equal to the objective 
value. If the difference of an actual cut exceeds 2i5|yl|,e.g. suppose it is (2i5+e)|yl|, 
then we can find an ordering with (^ + d + e/2)|v4| forward edges, which is a 
contradiction. This ordering is found by taking the cut that yields (2i5 + e)|A| 
more forward than backward edges and ordering the vertices in each of the two 
sets greedily so as to obtain at least half of the remaining edges. 

Suppose we have a set of unit vectors {ui} such that the value of equation 
(6) is at least {2S + e)|A| = fi\A\. We will show that we can find an actual cut 
such that the difference between the forward and the backward edges is at least 
(3\A\. Note that vq ■ Vi is a scalar quantity since vq is a unit vector that without 
loss of generality is (1, 0, 0, . . . ). Thus, our objective function can be written as 
~ "'^here 1 > Zi > —1. We transform the Zi variables into Xi 
variables that range between 0 and 1 by letting Zi = 2xi — 1. Then we have that 
~ ~ ~ This results in a linear program. We claim 

that an optimal solution to the following linear program is integral. 

^ {xj - X,) (7) 

ij^A 

0 < Xi < 1, \/i €V. 

To show this, consider rounding the variables by letting f be 1 with proba- 
bility Xi and 0 otherwise. Then the expected value of the solution is exactly the 
objective value. However, note that the value of the solution cannot be less than 
the expected value, since then there must exist a solution with value greater 
than the expected value, which contradicts the optimality of the expected value. 
Thus, the integral solution obtained must have difference of forward and back- 
ward edges that is equal to the objective value (7). □ 

We will also find a discussion of the following problem useful. Consider the 
problem of finding a balanced partition of the vertices of a given graph (i.e. each 
partition has size f ) that maximizes the weight of the edges that do not cross 
the cut. This problem is referred to as the max-^-uncut problem by Halperin 
and Zwick [HZOl]. Below is a integer quadratic program for the max-^-uncut 
problem. 

(T) 



max 



1 + Uj • Vj 



ij&A 
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Vi G {1, — 1} 
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Vi G V. 
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We obtain a semidefinite programming relaxation for the max-^-uncut problem 
by relaxing constraint (8) to: Vi e 7^", Vz. We denote the value of the relaxation 
of T on the graph G as Tr(G). 

Lemma 2. Let G = {V, A) and e, 5 he positive constants. Suppose the maximum 
acyclic subgraph of G is {^ + <5)|A|. If Pr{G) > (1 — e)|^|, then Tr{G) > 
{l-2e-2S)\A\. 



Proof. For each edge if G A, we have: 



E h i , h 
V, ■ Vj + n, 

h<i 



h-1 1-1 h 1-1 h-1 I 

Vj - V, ■ Vj - V, ■ Vj = 



(9) 
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W 




h<l 






(10) 


h< 






(11) 


?i>f ,f>f 






(12) 


h< -2 -X> -2 





For each edge, we refer to the quantity (9) as the forward value for that edge 
with respect to Pr{G). The same term summed instead over h > £ is referred to 
as the backward value of the edge with respect to Pr{G). We can simplify the 
terms above. Let Vi = vf . 

■ (^7 + ^o), 

h< 

~ ■ (^0 “ )’ 

h > -2 ,^> -2 

E K" - 

h< 

Since Pr{G) > (1 — e)|A|, we have: 

E E = E ^(^0 “ ■ (^0 + ^7) < e|A|. 

ijGA ij^A 



The above inequality says that the backward value of the vectors {ui} (i.e. quan- 
tity (5)) is at most the backward value of Pr{G). By Lemma 1, the difference 
of the edges crossing the cut in the forward direction and the edges crossing the 
cut in the backward direction is at most 2<5|A|. 




Cuts and Orderings 201 



+ ^ o ) ■ (^0 “ ■ (^0 + = 

ijGA ij^A 

X! • «0 - Vj ■ Vo) < 25\A\. 

ij&A 

This implies that the forward value cannot exceed the backward value by more 
than 2(5|A|. Thus, we can bound the forward value as follows: 

E E - E') = E • (^0 + ^ + 2^)i^i- 

ijGA h<^,£>^ ij^A 



This implies that if we sum the quantities (10) and (11) over all edges in A, then 
the total value of this sum is at least (1 — 2e — 2<5)|2l|. The sum of (10) and (11) 
taken over all the edges is: 



E lAVt- Vj 
2 

ij£A 



(13) 



□ 



3 Balanced Bisections of Random Graphs 

A bisection of a graph is a partition of the vertices into two equal (or with 
cardinality differing by one if n is odd) sets. We use a related definition in this 
section. 

Definition 1. A "f -bisection of a graph for 7 < 5 zs the set of edges that cross 
a cut in which each set of vertices has size at least yn. 

Suppose we choose an undirected random graph on n vertices in which every 
edge is present with probability p = ^. The expected degree of each vertex is 
2d and the expected number of edges is dn. We will call such a class of graphs 

Gp. 

Lemma 3 . For any fixed positive constants e,y, if we choose a graph G G Gp on 
n vertices for a sufficiently large n with p = — and d = w(l), then the minimum 
^-bisection contains at least (1 — 6 ) 7(1 — 'j)2nd edges with high probability. 

Proof. We will use the principle of deferred decisions. First, we will choose a 
yn, (1 — y)n partition of the vertices. Thus y(l — y)n^ edges from the complete 
graph on n vertices cross this cut. Then we can choose the random graph G by 
picking each edge with probability P = ^ . The expected number of edges from 
G crossing the cut is p = (y(l — y)n^)(^) = y(l — y)2c?n. For each edge in the 
complete graph that crosses the cut, we have the indicator random variable Xi 
such that Ai = 1 if the edge crosses the cut and = 0 if the edge does not 
cross the cut. Let A = 'YfXt, i.e. A is the random variable for the number of 
edges that cross the cut. By Chernoff Bound, we have: 



Pr[A < (1 — e)y(l — y)2dn)] < e 



-/(I — 'y)2dTi 
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We can union bound over all the possible 7-bisections. There are less than 2” 
ways to divide the vertices so that at least yn are in each set. Thus, the prob- 
ability that the minimum 7-bisection of G is less than a (1 — e) fraction of its 
expectation is: 

2'H. 

Pr[min 7-bisection (G) < (1 - 6)7(1 - -f) 2 nd] < ■ 



Let d = u;(l). Then for any fixed positive constants 7,6, this probability will be 
arbitrarily small for sufficiently large n. □ 



4 A Contradictory Cut 



In this section, we will prove our main theorem. Suppose we choose a directed 
random graph on n vertices in which every edge in the complete directed on n 
vertices is included with probability p. Let P = ^ and let d = w(l). We will call 
this class of graphs Gp. Note that if we randomly choose a graph from Gp, the 
underlying undirected graph is randomly chosen from Gp. 

Theorem 1. For sujficiently large n, d = and p = if we randomly 

choose a graph G G Gp, then with high probability, the ratio Pr{G) / P{G) < 
1.64. 



Let E represent the edges in the complete undirected graph for some fixed 
n. Let AGE represent the edges in an undirected graph G chosen at random 
from Gp. Let e\ be a small positive constant whose value can be arbitrarily small 
for sufficiently large n. We weight the edges in E as follows: 



n 

^ "(l-ei)2d 

Wij = 1 



if ij G A, 
if ij G E — A. 



We will refer to this weighted graph as G'. 



Lemma 4. The minimum ^-bisection of G' has negative value with high proba- 
bility. 



Proof. By Lemma 3 with high probability the minimum 7-bisection of G has at 
least (1 — 61)7(1 — 'y) 2 nd edges. Thus, with high probability the total weight of 
the edges in the minimum 7-bisection of G' is at most: 



7(1 - 7)n^ - (1 - 61)7(1 - j ) 2 nd + {I - 61)7(1 - j ) 2 nd {- -^ ei ) 2 d ^ ^ 

7(1 - 7 ) - (1 - 6i)2nd -f (1 - 6i)2nd(- -^i)2d ^ ^ 

7(1 - 7) (-(1 - ei)2nd) < 0. 



□ 
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Lemma 5. Let {fi}j i € V be a set of unit vectors that satisfy the following 
constraints: 

= 0 (14) 

i,j&V 

^ ( 1 - ^ 2 ) 1 ^ 1 - ( 15 ) 

ijGA 

If €2 < -36, then we can find a ^-bisection of G' with a strictly positive value. 
To prove Lemma 5, we will use the following theorem from [GW95]. 

Theorem 2.7 [GW] Let W- = Wijj where x = min(0,a:). Then 



- W-} > a i i E ^y(l “ - 11"- 

Proof of Lemma 5: We will use Goemans- Williamson’s random hyperplane 
algorithm to show that we can find a cut that is roughly balanced and has a 
strictly positive value. Let W represent the total weight of the edges that cross 
the cut obtained from a random hyperplane. Let W- denote the sum of the 
negative edges weights, i.e. W- = A. Applying Theorem 2.7 from [GW], we 
have: 





E[W] >a<-J2u!ij{^-Vi-Vj)-W-\+ W- 



i<j 



> a ■ 
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1- Vi - Vj 
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\wr 



A + Vi-Vj 



+ W_. 



i<j:wij >0 



i<j:wij <0 



We want to calculate the value of J2i<j wij>o By condition (14), we have 

that therefore J2i<j 

E l — Vi ■ Vj \ ^ 1 — Vi ■ Vj \ ^ 1 — Vi • Vj 

9 ~ 2 ^ 9 2 -^ 9 
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i<j-.Wij <0 



E l — Vi ' Vj nd 
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Now we have: 



E\W] > ai ( e2nd) + 



(1 - ei)2d 



(1 - e2)nd > - 



{l-ei)2d 



nd. 



For large enough n, we can choose e\ to be arbitrarily small. So iH[bF] can be 
bounded from below by a value arbitrarily close to the following: 



,OL OL 1 Ctt2\ 2 / 2\ ^ / 1 r-r,r- 2 / 2\ 

-o(n) > (.1585 -— )n - o(n ). (16) 



If the value of 62 is such that the quantity on line (1 6) is strictly greater than /3n^ 
for some positive constant (3, then we will have a contradiction for sufficiently 
large n. Note that if this value is at least I3n?, then each side of the cut contains 
at least ^/]3n vertices, so it is a v^-bisection. This value will be strictly positive 
as long as £2 < .36. Thus, it must be the case that £2 > .36. □ 

Proof of Theorem 1: We fix positive constants 7 ,£i. Suppose we choose a 
random directed graph G as prescribed and let the graph G = {V, A) be the 
undirected graph corresponding to the underlying undirected graph of G. We 
then weight the edges in the graph AT„ as discussed previously and obtain G' . 
By Lemma 4, the minimum 7 -bisection of G' is negative with high probability. 
Thus, with high probability equation (15) hold only when £2 > .36. 

Suppose the maximum acyclic subgraph of G, i.e. P{G) is -I- < 5 )| 4 l| for 
some positive constant 6. Then the value of Pr{G) is upper bounded by the 
maximum value for some set of unit vectors {ui}of ( 10 ), ( 11 ), and ( 12 ) summed 
over all edges in A. Note that this is equivalent to the quantity in (13) (which 
is no more than .64|A|) plus the quantity in (4). By Lemma 1, the difference 
between (4) and (5) must be no more than 2i5|7l|. Thus, we can upper bound 
the value of Pr{G) by .64|A| -I- {26 + ^(.36 — 2<5))|A| = (.82 -I- 5)|A|. Thus, with 
high probability, we have: 



Pr{G) ^ .82 + 6 ^ 
P{G) - .b + 6 - 



.82 



= 1.64. 



□ 



5 Discussion 

In this paper, we make a connection between cuts and vertex ordering of graphs 
in order to obtain a new semidefinite programming relaxation for the linear 
ordering problem. We show that the relaxation is “good” on random graphs 
chosen with uniform edge probability, i.e. the integrality gap is strictly less than 
2 for most of these graphs. We note that we can extend this theorem to show 
that this relaxation is “good” on graphs that have no small 7 -bisections for some 
constant 7 > 0 . 

In [HZOl], Halperin and Zwick give a .8118-approximation for a related prob- 
lem that they call the max ^-directed-uncut problem. Given a directed graph, 
the goal of this problem is to find a bisection of the vertices that maximizes the 
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weight of the edges that cross the cut in the forward direction plus the weight of 
the edges that do not cross the cut. We note that a weaker version of Theorem 1 
follows from their .8118-approximation algorithm. This is because their semidef- 
inite program for the max ^-directed uncut problem is the sum over all edges 
of terms (10), (11), and (12). If for some directed graph G = {V,A), Pr{G) has 
value at least (1 — e)|^|, then the value of their semidefinite programmming re- 
laxation also has at least this value. Thus, if e is arbitrarily small, we can obtain 
a directed uncut of value close to .8118 of the edges, which is a contradiction for 
a random graph with uniform edge probability. In this paper, our goal was to 
give a self-contained proof of this theorem. 

We would like to comment on the similarity of this work to the work of Pol- 
jak and Delorme [DP93] and Poljak and Rendl [PR95] on the maxcut problem. 
Poljak showed that the class of random graphs with uniform edge probability 
could be used to demonstrate an integrality gap of 2 for several well-studied 
polyhedral relaxations for the maxcut problem [Pol92]. These same graphs can 
be used to demonstrate an integrality gap of 2 for several widely-studied polyhe- 
dral relaxations for the linear ordering problem [NVOl]. The similarity of these 
results stems from the fact that the polyhedral relaxations for the maxcut prob- 
lem are based on odd-cycle inequalities and the polyhedral relaxations for the 
linear ordering problem are based on cycle inequalities. Poljak and Delorme 
subsequently studied an eigenvalue bound for the maxcut problem that is equiv- 
alent to the bound provided by the semidefinite programming relaxation used 
in the Goemans-Williamson algorithm [GW95]. Despite the fact that random 
graphs with uniform edge probability exhibit worst-case behaviour for several 
polyhedral relaxations for the maxcut problem, Delorme and Poljak [DP93] and 
Poljak and Rendl [PR95] experimentally showed that the eigenvalue bound pro- 
vides a “good” bound on the value of the maxcut for these graphs. This exper- 
imental evidence was the basis for their conjecture that the 5-cycle exhibited 
a worst-case integrality gap of 0.88445 . . . for the maxcut semidefinite relax- 
ation [DP93,Pol92]. The gap demonstrated for the 5-cycle turned out to be very 
close to the true integrality gap of .87856 . . . [FS]. 
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Abstract. We propose the Min-max multiway cut problem, a variant 
of the traditional Multiway cut problem, but with the goal of mini- 
mizing the maximum capacity (rather than the sum or average capacity) 
leaving a part of the partition. The problem is motivated by data parti- 
tioning in Peer-to-Peer networks. The min-max objective function forces 
the solution not to overload any given terminal, and hence may lead to 
better solution quality. 

We prove that the Min-max multiway cut is NP-hard even on trees, 
or with only a constant number of terminals. Our main result is an 
0(log® n)-approximation algorithm for general graphs, and an 0(log^ n)- 
approximation for graphs excluding any fixed graph as a minor (e.g., 
planar graphs). We also give a (2 -|- e)-approximation algorithm for the 
special case of graphs with bounded treewidth. 



1 Introduction 

The Min-max multiway cut problem is defined by an undirected graph G = 
(V,E) with edge capacities c(e) > 0, and a set A = {x\,...,Xk} C U of distin- 
guished nodes called terminals. A multiway cut is a partition of V into disjoint 
sets Si , ..., Sk (Ui Si = V), so that for all i G {1, ..., k}, Xi G Si. For a partition 
we will use S{Si) to denote the capacity of the cut separating Si from the other 
sets Ujyii Sj , and the goal of the min-max multiway cut problem is to minimize 
the maximum capacity max^ 6{Si). 

The min-max multiway cut problem models the data placement problem 
in a distributed database system or a Peer-to-Peer system. In a Peer-to-Peer 
database, the information is stored on many servers. When a user query is issued, 
it is directed to the appropriate server. A request for some data item v can 
lead to further requests for other data. One important issue in such Peer-to- 
Peer databases is to find a good distribution of data that minimizes requests to 
any single server. We model this by a graph in which the non-terminal nodes 
represent the data items and the terminals represent the servers. Nodes in the 
partition Si correspond to the data that will be stored on server i. Edges in the 
graph correspond to the expected communication patterns, i.e., the edge (xi,v) 
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represents the number of queries that users at server i issue for the data v, and 
the edge {v, w) represents the expected number of times that a request for data 
V will result in an induced request for data w. Communication costs are incurred 
when a query from one server is sent to another. The goal then is to distribute 
the data among the servers so as to minimize the communication cost incurred 
by any one of them. 

The min-max multiway cut problem is closely related to the traditional mul- 
tiway cut problem of [2]. The difference is in the objective function. Unlike the 
min-max multiway cut, in which we seek to minimize the maximum capacity 
maxii5(S'i), the multiway cut problem evaluates a partition by the sum of the 
capacities of all edges that connect the parts, thus minimizing the average ca- 
pacity S{Si). Multiway cut has been used to model similar applications of storing 
files on a network, as well as other problems such as partitioning circuit elements 
among different chips [2]. In many situations, however, the min-max objective 
function may be a better representation of the solution quality. Although the 
multiway cut minimizes the average communication cost of the terminals, this 
cost may not be distributed uniformly among them, resulting in a very heavy 
load on some terminals and almost no load on others. The objective of mini- 
mizing the maximum load tries to alleviate this problem by ensuring that no 
terminal is overloaded. 

Multiway cut problem is NP-hard, but there are very good approximation 
algorithms for it [2], [1], [6]. However, they do not translate directly into good 
approximations for min-max multiway cut, because even the optimal solution to 
one problem can be up to a factor of k/2 worse than the optimum for the other. 

Our Results. For two terminals, min-max multiway cut reduces to the well- 
studied minimum s-t cut problem, and hence it can be solved in polynomial 
time. However, as we show, it is already NP-hard for the case of 4 terminals. As a 
result, we focus on designing approximation algorithms. In Section 2, we present 
an 0{a • log n)-approximation algorithm for min-max multiway cut, where a = 
log^ n for general graphs, and a = logn for graphs excluding any fixed graph as 
a minor. The algorithm uses an a-approximation algorithm for a new graph cut 
problem, called Maximum size bounded capacity cut (MaxSBCC). We use 
it as a subroutine in a procedure that resembles the greedy set cover algorithm, 
incurring an additional factor of 0(log ri) in the approximation guarantee for the 
min-max multiway cut problem. One of the features of our algorithm is that it 
is able to exhibit flexibility when assigning graph nodes to terminals: if the cut 
that is found for one terminal is later discovered to be bad for another terminal, 
then the nodes are reassigned in a way that is good for both. 

We extend our algorithm to a generalization of the problem, in which there 
is a separate bound Bi for each terminal a;^, and the goal is to find a partition in 
which 6{Si) does not exceed Bi. This generalization is useful when the different 
peers corresponding to the terminals have different communication capabilities, 
and can withstand different loads. 

Turning to special cases of min-max multiway cut, we show that it is NP- 
complete even on trees, and develop a (2 -|- e)-approximation algorithm for trees 
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and graphs with bounded treewidth. What makes the problem hard on trees is 
that an optimal solution does not necessarily assign connected components of the 
tree to each terminal (see Figure 3, in which the black nodes are the terminals, 
and the optimal solution must assign the white node in the middle to one of 
the leaves). As a result, even if we know which edges should be cut, it may be 
hard to determine how to divide the resulting components among the terminals. 
The key idea of our (2 -|- e) approximation algorithm is to separate the stage of 
finding connected pieces of the graph from the stage of partitioning them among 
the terminals. Then, in the first stage, the problem of finding “good” pieces is 
solved optimally, and in the second stage these pieces are combined to form a 
2- approximate solution. To make the dynamic programming algorithm of the 
first stage run in polynomial time, the edge capacities are rounded, leading to 
an overall {2 + e) -approximation. 

2 Min-Max Multiway Cut in General Graphs 

2.1 Approximation Algorithm 

Our main goal in this section is to provide an approximation algorithm for the 
min-max multiway cut problem and its extension with nonuniform bounds on 
the capacities. 

First we briefly recall the 2-approximation algorithm of Dahlhaus et al. [2], 
as it is useful to understand why it does not work for the min-max version of 
the problem. Assume that there is a multiway cut with maximum capacity at 
most B. The algorithm finds a minimum capacity cut (Si,Ti) separating each 
terminal Xi from all other terminals. It is not hard to see that the minimum 
cuts with smallest source sides Si are disjoint. Let So be the nodes not in any 
Si, and let Si = 6{Si) be the capacity of the cut (Si,Ti). The cut (Si,Ti) is of 
minimum capacity, so we must have Si < B. The algorithm of [2] assigns each 
set Si to terminal Xi , and assigns the remaining nodes Sq to one of the terminals 
(the one with maximum Si). This yields a 2-approximation (or, more precisely, 
a 2(1 — 1/fc) approximation) algorithm for the multiway cut problem, but it is 
only a, (k— l)-approximation for the min-max multiway cut problem, as the part 
Si U So can have capacity as high as ^i^j) < (^ ~ 1)^- 

The idea of our algorithm is to take cuts around each terminal that are larger 
in size than the minimum cut. Assume that we are given a bound B, and assume 
that there is a multiway cut where each side has capacity at most B. We will 
use a binary search scheme to optimize B. For a given value of B, we will need 
a subroutine for the following maximum size bounded capacity cut problem. 

Definition 1. Given a graph G = (V,E) with two distinguished vertices s and 
t, weights on vertices w{v), capacities on edges c(e), and an integer B, the Max- 
imum SIZE BOUNDED CAPACITY CUT (MaxSBCC) problem is to find an s-t cut 
(S,T) such that S{S) < B and w{S) = maximized. 

The MaxSBCC problem can be shown to be NP-hard using a reduction from 
Knapsack. For a > 1 and a constant 0 < /3 < 1, let us define an {a, (3)- 
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approximation algorithm for MaxSBCC as an algorithm that, given an instance 
of MaxSBCC with a bound B and an (unknown) optimal solution (S*,T*), 
produces in polynomial time an s-t cut {S',T'), such that S{S') < aB and 
w{S') > f3w{S*). 

First we show how to use any such approximation algorithm as a subroutine 
for solving the min-max multiway cut problem, and later we give a specific 
{log^n, 1) algorithm. The idea is analogous to the greedy log n-approximation for 
the set-cover problem. Starting from the set V of unassigned nodes of the graph, 
our algorithm iteratively finds (approximate) maximum size bounded capacity 
cuts around each terminal, and temporarily assigns nodes to terminals, until no 
unassigned nodes remain. One important difference is that our algorithm is not 
greedy, in the sense that assignment made to terminals in one iteration can be 
revised in later iterations if that becomes useful. The full algorithm is shown in 
Figure 1. 



1. Initialize Si = {xi} for i = 1, ..., k, and initialize the weights w(v) for all n G F by 
setting w(xi) = 0 for all i, and w{v) = 1 for all other nodes. 

2. While U . Si 7 ^ F 

— For each terminal Xi £ X, 

(a) Construct a graph G' labeling Xi as source s and contracting all other 
terminals into a single sink t. 

(b) Find an (a, /3)-approximate MaxSBCC {S, T) in graph G' with bound B 
and weights w{v). Note that the set S does not have to contain Si and 
does not have to be disjoint from the other sets Sj for j ^ i. 

(c) Consider the intersection Ij = S C] Sj for each j 7 ^ i. We need to delete 
this intersection either from Sj or from S. If c{Ij, Sj \ I j) < c{Ij, S \ Ij), 
then let Sj = Sj \ Ij ; otherwise let S = S \ Ij . 

(d) Let Si = Si U S, and set the weights of all u G S to w(v) = 0. 

3. Return Si, ..., Sfc. 



Fig. 1. Min-max multiway cut algorithm 



Theorem 1. If there is an {a, (3) -approximation algorithm for MaxSBCC, then 
the above algorithm is an O{alogi^^n)-approximation for the Min-MAX mul- 
tiway CUT problem. 

The key to the analysis is to see that each iteration assigns a constant fraction 
of the remaining nodes. By assumption there is a multiway cut {SI,..., SI) with 
capacity B. For each terminal Xj, we use the approximation bound to claim that 
the application of the MaxSBCC assigned at least as many new nodes to Xi as 
a j3 fraction of the remaining nodes in S* . 



Lemma 1. If there is a multiway cut with maximum capacity at most B, then 
in any iteration of the while loop, if U is the set of unassigned nodes in the 
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beginning of the iteration, and U' is the set of unassigned nodes at the end of 
this iteration, then \U'\< j^\U\. 

Proof. Let Ni be the set of previously unassigned nodes added to the set Si in 
this iteration. Notice that step (2c) of the algorithm only reassigns nodes with 
zero weight, so Ni has the same weight as the solution to MaxSBCC S obtained 
in step (2b). 

Consider some optimal solution S’^) to the min-max multiway cut 

instance. Now partition U' into sets U[, U^, such that U[ = We claim 

that w{Ni) > fd ■ \U[\. To see this, notice that the nodes in U' have weight 1 
throughout this iteration of the while loop, and since S* is a piece of the optimal 
partition, S{S*) < B. Therefore, in the iteration of the for loop, {S* ,V\S*) 
is a feasible solution to the MaxSBCC problem, and w{S*) > w{Ul) = |C/'|. 
By the (a, /3)-approximation guarantee, the algorithm for MaxSBCC must find 
a set with w{Ni) > fd ■ |17'|. Summing over all i, we obtain that \U\ — \U'\ = 
^i=i w{Ni) > fd ■ \U'\, which proves the claim. □ 

Proof (of Theorem 1). By using binary search, we can assume that a bound B 
is given, and our algorithm will either prove that no multiway cut of maximum 
capacity at most B exists, or it will find a multiway cut with capacity at most 
0(alogi+^n)B. 

Throughout the algorithm, Xi G Si for all i, and the sets Si are always 
disjoint. So the algorithm finds a multiway cut, as required. By Lemma 1 the 
algorithm terminates in at most log]^_|_^ n iterations of the while loop, if a min- 
max multiway cut of capacity at most B exists. If given an infeasible bound 
B < B*, it may not stop after log;^_|_^ n iterations, which proves that B < B* . 
This shows that the algorithm runs in polynomial time. We will also use this 
bound to give an approximation guarantee for the algorithm. 

We claim that for each Si returned by the algorithm, 6{Si) < alog^j^p n ■ B. 
To see this, notice that for each application of the MaxSBCC subroutine in (2b), 
the capacity of the set S returned is at most 6{S) < aB. By the choice made 
in step (2c), the transfer operation does not increase either (5(Sj) or S{S). So in 
each iteration of the while loop, the capacity of each Si increases by at most 
aB. Combined with the bound on the number of iterations, this observation 
concludes the proof. □ 

Feige and Krauthgamer [3] give an 0(log^ n) approximation algorithm for the 
problem of finding cuts with specified number of nodes, and an improved 0(log n) 
approximation for the case when the input graph G is assumed not to contain 
a fixed graph as a minor (e.g., for planar graphs). We will use this algorithm 
to give an (0(log^ n), 1) approximation algorithm for the MaxSBCC problem in 
general graphs and an improved (0(logn),l) approximation algorithm in the 
special case. This will yield the following theorem. 

Theorem 2. There is an O {log^ n)- approximation algorithm for Min-MAX 
MULTIWAY CUT problem on general graphs, and an O (log^ n)- approximation for 
graphs excluding any fixed graph as a minor (e.g., planar graphs). 
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Proof. Feige and Krauthgamer [3] give an algorithm for finding cuts with spec- 
ified sizes. For a graph G with n nodes and each number d < n, their algorithm 
finds a cut {Sd,Td) with \Sd\ = d and capacity 6{Sd) within a = 0(log^ n) of 
the minimum capacity for such a cut. For graphs excluding any fixed graph as 
a minor, this guarantee is improved to a' = O(logn). The algorithm also works 
for finding s-t cuts on graphs with node weights and edge capacities. 

We claim that the cut that corresponds to the largest value d* such that 
S{Sd*) < aB is an (a, l)-approximate MaxSBCC. By definition, its capacity 
is at most aB. And, if the optimal MaxSBCC had size d' > d* , then, by the 
guarantee of the algorithm, 5{Sd') would be at most aB, contradicting our choice 
of d* . The result then follows from Theorem 1. □ 

It is interesting to note that the algorithm can also be used for a version of 
the multiway cut problem in which there is a separate bound Bi for each 5{Si). 
To obtain the extension, we use the MaxSBCC algorithm in each iteration i of 
the for loop with bound Bi rather than B. 

Theorem 3. Assume we are given a graph G with k terminals, edge capacities, 
and k hounds {B\, . . . ,B}f). If there is a multiway cut {Si, . . . , Sk) such that 
h{Si) < Bi for each i, then in polynomial time we can find a multiway cut 
{S'l, . . . , S'jf) such that <5(5'') < 0(log^ n)Bi, and the bound improves by a factor 
of log n for graphs excluding any fixed graph as a minor. 

Remark. Calinescu, Karloff and Rabani [1] and subsequently Karger et al. [6] 
gave improved approximation algorithms for the multiway cut problem based on 
linear programming and rounding. It appears that this technique does not yield 
a good approximation for our problem. To see this, consider the graph which is 
a star with k terminals and a single additional node at the center, and assume 
the capacity of each edge is 1. There is no multiway cut where each part has 
capacity at most B = 2, or even approximately 2. By assigning the center of the 
star to terminal Xi, we can get a multiway cut where the capacity of each part Sj 
for j yf i is 1, while the capacity of S'i is A: — 1. A linear programming relaxation 
would allow us to take a “linear combination” of these cuts, and thereby have 
each side have capacity at most 2. 



2.2 NP- Completeness 

We prove using a reduction from Bisection that the Min-max multiway cut 
problem is NP-hard already on graphs with 4 terminals. 

Theorem 4. Min-max multiway cut is NP-hard for any fixed k > 4 even 
with unit-capacity edges. 



Proof. We will show that it is NP-hard for k = 4 using a reduction from Bisec- 
tion [4]. Our construction uses capacities, but we can replace each edge with 
multiple parallel paths. An instance of Bisection consists of a graph G = {V, E) 
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2na . 




Fig. 2. Reduction from Bisection to Min-max multiway cut 



with an even number of vertices n, and an integer C . The question is whether 
or not there exists a partition of V into two sets X and Y , each of size n/2, 
such that the capacity of the cut c(X, Y) < C. Given G and C, we construct, in 
polynomial time, a graph F with 4 terminals and a bound B, so that F has a 
multiway cut with maximum capacity at most B if and only if G has a bisection 
with capacity at most C. 

We obtain the graph F = {V',E') by adding 4 new terminal nodes X = 
{u, d, I, r} to G, and adding edges that connect nodes of G to the terminals (see 
Figure 2). E' includes E and the following additional edges, where a is chosen 
such that 2a > C: 

— Edge (m, d) of capacity na 

— Edges {v, u) and {v, d), each of capacity a, for each v gV. 

— Edges (u, 1) and (u, r), each of capacity 5 = a — ^ > 0, for each v G V. 

The bound is set to B = 2na. 

Suppose F has a min-max multiway cut {U U {«}, D U {d}, L U {/}, i? U {r}) 
where each part has capacity at most B. Then U and D must be empty, as 
B = 2na > 5{U U {m}) > 2na+ 2fo|C/|, just counting the edges to the terminals. 
So (L, R) is a cut of G, and let G' = c(T, i?) denote its capacity. The next 
observation is that \L\ = |i?| = n/2. To see this, suppose, for contradiction, that 
\L\ = k> ^ + 1 (or similarly for \R\). Then 

n G 

6(L U {/}) = 2ka + nb + G' > 2(— + l)a -I- n(a ) = 2na + (2a — G) > B, 

2 n 

where the last inequality follows from the choice of a. We conclude that the 
capacity G' of the bisection (L, R) must be at most G. This follows as the 
capacity of the cut L U {/} is na -I- n6 -|- G' < B = 2na, and by the choice 
of b this inequality implies G' < G. To show the opposite direction, given a 
bisection (X, F) in G of capacity G' < G, we produce a min-max multiway cut 
({n}, {d}, XU {/}, FU {r}) of E, with each component’s capacity at most B. □ 
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3 Min-Max Multiway Cut on Trees 
and Bounded Treewidth Graphs 

Recall from the Introduction that in an optimal solution to the min-max multi- 
way cut problem on trees the sets of nodes assigned to the terminals do not have 
to be connected. This can be seen in the example of Figure 3. All nodes except 
for the middle one are terminals, and all edges have capacity 1. The optimal 
solution cuts all the edges incident on the middle node and assigns it to one of 
the leaf (degree-one) terminals, achieving a value of 4. On the other hand, any 
solution that assigns connected parts of the graph to each terminal would leave 
the middle node connected to one of its neighbors, incurring a cost of 5. 




Fig. 3. Example showing that in an optimal min-max multiway cut on a tree, the sets 
assigned to the terminals need not form connected components. The only non-terminal 
in this graph is the middle node 



In Section 3.1 we use this observation to prove that the Min-max multiway 
CUT problem in NP-hard on trees. Then we provide a (2 -|- e)-approximation on 
trees, and, finally, in Section 3.4 we extend it to graphs with bounded treewidth. 

3.1 NP- Completeness 

Theorem 5. Min-max multiway cut is strongly NP-hard when the graph is 
a tree with weighted edges. 

Proof. We use a reduction from 3-partition, which is known to be strongly 
NP-complete [5]. In 3-partition, given a set A = {ai, ..., asm }, a weight Wi for 
each Oi € A, and a bound B, such that 'ii B jA < Wi < B f2 and = mB, 

we want to know if A can be partitioned into disjoint sets Si , ..., Sm, such that 
for each j, 



Wi = B. 

di G S j 



Given an instance (A, B) of 3-partition, we construct an instance of min-max 
multiway cut as follows. The tree T consists of separate subtrees connected with 
zero-capacity edges. There will be 3m subtrees Ti, one for each element a^, and 
m isolated terminals xi , ..., Xm, one for each of the desired sets. Each Ti consists 
of six terminals and one non-terminal Vi, with edge capacities as in Figure 4. 
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B-Wi/3 Wi/3 

m • 




Fig. 4. Component Ti used in the NP-completeness reduction for Min-max multiway 
CUT on trees 

We claim that a min-max multiway cut of maximum capacity at most B 
exists if and only if the 3-partition instance is solvable. Notice that any min- 
max multiway cut with capacity at most B must cut all edges of Ti and assign 
all Vi’s to the terminals xi, Xm, creating a partition of A. If a set of nodes 
is assigned to terminal xj, then the capacity of the resulting part is gS' 
This implies that such a cut exists if and only if the 3-partition does. □ 

3.2 Algorithm for Min-Max Multiway Cut on Trees 

In this section we give a (2 -|- e)-approximation algorithm for the min-max mul- 
tiway cut on trees that have edge capacities. 

The algorithm consists of two stages. In the first stage we consider a variant 
of the problem where we allow the algorithm to create extra parts in the partition 
that do not contain terminals. More precisely, we consider the following problem. 

Definition 2. The Tree cutting problem (T,X,B) for a tree T = {V,E), 
terminals X = {xi,...,a;fc} C V, and a bound B is to find a partition of V 
into connected subtrees subject to the following constraints: (1) no 

two terminals are in the same connected component; and (2) for each connected 
component Ti, d(Ti) < B. The objective is to minimize ^iS(Ti). 

In the next subsection we give a pseudo-polynomial time algorithm for this 
problem. Here we show how to use such an algorithm to get a (2 -|- e)-approxi- 
mation for the min-max multiway cut on trees. 

Theorem 6. Using a pseudo-polynomial time algorithm for the Tree cutting 
problem as a subroutine, we can give a polynomial time (2 -|- e)- approximation 
for the Min-max multiway cut on trees. 

Proof. First we give a 2-approximation for min-max multiway cut that uses a 
pseudo-polynomial exact algorithm for tree cutting. Given a tree T with termi- 
nals X , we will binary search for a lower bound B* to the min-max multiway cut 
optimum. Observe that connected components in a feasible solution to min-max 
multiway cut instance (T,X) of value B give a feasible solution to the tree cut- 
ting instance (T, X, B) of value at most kB. Therefore, if our optimal tree cutting 
solution Ti,...,Th does not satisfy J2i=iK^i) ^ then B < B*. The algo- 
rithm groups the components Ti into k sets Si, ..., Sk of nodes greedily, assigning 
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the terminals to different sets. Observe that if several components, say T\, Tj, 
are combined into a set S, then 6{S) < X)i=i S{Ti). Because < kB, 

and each S{Si) is at least B, all components will be packed into the k sets. Also, 
since for all j, S{Tj) < B, for no i will 6{Si) exceed 2B. 

Recall that our tree cutting algorithm runs in pseudopolynomial time. We 
obtain a polynomial-time (2 -|- e)-approximation algorithm via rounding. For a 
given e, capacity bound B, and m = \E\, let 



For each edge e G E, scale the capacity so that c'(e) = [ac{e)\ . Also set B' = 
aB = [m/e] . If there is a multiway cut with maximum capacity B in the original 
problem, then there is one of maximum capacity at most B' after rounding. Now 
we obtain a 2-approximate multiway cut Si, Sk for the graph with capacities 
c'(e) and bound B' . The running time is polynomial in B' = |"m/e] and n. The 
capacity of a part Si of this partition is at most 5 {Si) < {2 + e) ■ B using the 
original capacities. □ 



3.3 Algorithm for the Tree Cutting Problem 

We now describe an algorithm that solves optimally, in time polynomial in B and 
the size of the tree n, the tree cutting problem (which we used as a subroutine 
for the min-max multiway cut on trees). To simplify the presentation of the 
algorithm, assume, without loss of generality, that (1) T is rooted at a node r 
and all edges are directed away from the root; (2) T is binary. (To make the tree 
binary without affecting the solution, replace each node u that has d > 2 children 
with a [log 2 d— 1] -height complete binary subtree U with edge capacities B+1, 
and attach u’s children to the leaves of U, at most 2 per leaf.) 

The tree cutting problem will be solved using dynamic programming. First 
consider the simpler problem with no terminals. To solve this problem, we con- 
struct a dynamic programming table p{v, A) for all nodes v G V and integers 
0 < A < B, where the entry p{v, A) is the minimum total capacity of edges in 
the subtree of T rooted at v that can be cut such that the total capacity of edges 
coming out (i.e., toward descendants) of u’s component is at most A. We have 
the separate bound A because the remaining B — A capacity will be used to cut 
the edges that are incident on u’s component, but lie outside of its subtree. The 
values p{v, A) can be computed in a single pass up the tree. If a node u has one 
child ui, then cutting (u,ui) implies that the component containing ui can have 
at most B — c(u,ui) capacity below ui, so the total capacity obtained this way 
is c(u,ui) +p{vi,B — c(u,ui)). If we do not cut the edge (u,ui), then we get 
p{vi,A). This leads to the following recurrence for the case that u has a single 
child ui. 



p{v,A) 



p{vi,A) if c(u,ui) > A 

min{p(ui. A), c(u, ui) +p(vi,B — c(u,ui))} otherwise. 
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Now suppose that the internal node v has two children, v\ and V 2 - Then the 
capacity A available for cutting edges below v has to be partitioned between the 
edges that belong to the left subtree (including, possibly, (u,ui)), and the ones 
that belong to the right subtree (possibly including (^,^ 2 )). The algorithm tries 
all possibilities for such a partition Ai + A 2 = A. Then, given Ai, it decides 
independently for each child node Vi whether or not to cut (n,Ui), using an 
expression similar to the one above. 

Next we extend the algorithm to make sure that all terminals in the tree are 
separated. For this, a binary variable t is added to the parameters of the table. 
The value of t limits the options available to the above simpler algorithm in each 
step. It will either require that a given component not contain a terminal (t=0), 
or it will not impose such a restriction {t = *). The idea is that if a node is 
connected to some ancestor which is a terminal, then it may not be connected 
to any descendants which are terminals, so in this case we will use a table entry 
with t = 0. Also, care has to be taken that a node is not connected to two 
terminals which are both descendants. 

Theorem 7. The optimal solution to the Tree cutting problem can be com- 
puted in time polynomial in the size of the graph and the bound B. 

3.4 Bounded Treewidth Graphs 

Finally, we extend the (2 -|- e)-approximation algorithm for min-max multiway 
cut to work on graphs with bounded treewidth (see [7] for an introduction to 
tree decomposition). The only change is in solving the tree cutting problem. 

First we note that the algorithm from Section 3.3 can work on trees with 
degree greater than two. The only potential difficulty is to generate the optimal 
guesses of Ax,..., Ad, ^Ai = A, in polynomial time. Given the values of the 
subproblems at the leaves, the problem of finding the best partition to obtain the 
value p{v, A) is essentially a knapsack problem, and hence is solved by optimally 
solving the problem for each suffix j, ..., d of the set of node’s children, with j 
going from d to 1, and using the optimal result for j, ..., d in order to solve the 
problem for j — 1, ..., d. 

Now we sketch how to extend this algorithm to handle graphs with bounded 
treewidth. Suppose that we are given a graph G = {V, E) and a decomposition 
tree T for it, such that for each node u in T, there is a set Vu C V associated with 
it. The size of each Vu is bounded by a constant b. Let us root T at some node r 
and assign a height h{u) to each node u G T so that r is the highest. Now we can 
associate with each vertex v G V a, label, which is defined as max{h{u)\v G Vu}. 
The algorithm will again build a dynamic programming table. In this case a 
table entry will be p{u, {Hi, ..., Hh}, [Ax, ..., Ah}, {tx, ..., th}), where u is a node 
in T; Hi’s form a partition of Vu that has h components, for some h G [1, ...,6], 
with the meaning that different Hi’s will be subsets of different components in 
the solution; Ai is a bound on the total capacity of edges (v,w) such that v 
is in the same component as Hi, w is not, and label{v) > label (w); and ti is 
a variable for Hi that specifies, as before, whether this component is allowed 
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to contain terminals. The computation proceeds in a bottom-up fashion on the 
tree T. When a node u of T and its child node ui are considered, all allowed 
combinations of vertices in Vu and are evaluated, subject to the constraints 
imposed by Hi’s and tj’s, and the best one is chosen. If u has multiple children, 
then each Ai is divided among the children in the same way as before. The 
running time of the algorithm is exponential in b, but, given that 6 is a fixed 
constant, it remains polynomial (or pseudopolynomial, as before). 

Theorem 8. The above algorithm computes the optimal solution to the Tree 
CUTTING problem in graphs with bounded treewidth in time polynomial in the size 
of the graph and the bound B. As a consequence, we get a {2 + e)- approximation 
for the Min-max multiway cut in graphs with bounded treewidth. 
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Abstract. Given any integer d > 3, let k be the smallest integer such 
that d < 2fclogfe. We prove that with high probability the chromatic 
number of a random d-regular graph isfc, fc + 1, orfc + 2. 



1 Introduction 



In [10], Luczak proved that for every real d > 0 there exists an integer k = k{d) 
such that w.h.p.^ x(t/(n,d/n)) is either fc or fc+ 1. Recently, these two possible 
values were determined by the first author and Naor [4]. 

Significantly less is known for random d-regular graphs Qn,d- In [6], Frieze 
and Luczak extended the results of [9] for xiSin-tP)) to random d-regular graphs, 
proving that for all integers d > do, w.h.p. 



X{Gn,d) 



d ^ f d log log d \ 

21ogd " “ V (logd)2 ) 



Here we determine x(Gn,d) up to three possible values for all integers. More- 
over, for roughly half of all integers we determine x{Sn,d) up to two possible 
values. We first replicate the argument in [10] to prove 

Theorem 1. For every integer d, there exists an integer k = k{d) such that 
w.h.p. the chromatic number of Gn,d is either k or k + 1. 

We then use the second moment method to prove the following. 

Theorem 2. For every integer d, w.h.p. x{0n,d) is either k, k+1, ork+2, where 
k is the smallest integer such that d < 2k log k. If, furthermore, d > (2k— 1) log k, 
then w.h.p. x(Gn,d) is either k + 1 or k + 2. 

The table below gives the possible values of x{Gn,d) for some values of d. 



d 


4 


5 


6 


7,8,9 10 


100 


1,000,000 


X{Gn,d) 


3,4 


3,4,5 


4,5 


4,5,6 5,6 


18,19,20 


46523, 46524 



^ Given a sequence of events S„, we say that S holds with positive probability (w.p.p.) if 
liminf„^/oo Pr[fn] > 0, and with high probability (w.h.p.) if liminf„^oo Pr[fn] = 1. 



K. Jansen et al. (Eds.): APPROX and RANDOM 2004, LNCS 3122, pp. 219-228, 2004. 
Springer- Verlag Berlin Heidelberg 2004 
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1.1 Preliminaries and Outline of the Proof 

Rather than proving our results for Qn,d directly, it will be convenient to work 
with random d-regular multigraphs, in the sense of the configuration model [5]; 
that is, multigraphs Cn,d generated by selecting a uniformly random configura- 
tion (matching) on dn “vertex copies.” It is well-known that for any fixed integer 
d, a random such multigraph is simple w.p.p. As a result, to prove Theorem 1 
we simply establish its assertion for Cn,d- 

To prove Theorem 2 we use the second moment method to show 

Theorem 3. If d < 2fclogfc, then w.p.p. x{Cn,d) < fc-|- 1. 

Proof of Theorem 2. For integer k let Uk = (2fc — l)logfc and Ck = 2fclogfc. 
Observe that Ck-i < Uk < Cfe. Thus, if k is the smallest integer such that d < Ck, 
then either \) Uk < d < Ck or ii) Uk-i < c^-i < d < < Ck. 

A simple first moment argument (see e.g. [11]) implies that if d > Uk then 
w.h.p. x{Cn,d) > k. Thus, if Uk < d < Ck, then w.h.p. Cn,d is non-fc-colorable 
while w.p.p. it is (fc -I- l)-colorable. Therefore, by Theorem 1, w.h.p. the chro- 
matic number of Cn,d (and therefore Qn,d) is either /c -|- 1 or /c -|- 2. In the second 
case, we cannot eliminate the possibility that Qn,d is w.p.p. fc-colorable, but we 
do know that it is w.h.p. non-(/c — l)-colorable. Thus, similarly, it follows that 
x{Gn,d) is w.h.p. k,k+l or k + 2. □ 



Throughout the rest of the paper, unless we explicitly say otherwise, we 
are referring to random multigraphs Cn,d- We will say that a multigraph is k- 
colorable iff the underlying simple graph is fc-colorable. Also, we will refer to 
multigraphs and configurations interchangeably using whichever form is most 
convenient. 

2 2-Point Concentration 

In [10], Luczak in fact established two-point concentration for y(C/(n, d/n)) for 
all e > 0 and d = 0(n^/®“'^). Here, mimicking his proof, we establish two-point 
concentration for x{Gn,d) for all e > 0 and d = 0(n^/^“*^). 

Our main technical tool is the following martingale-based concentration in- 
equality for random variables defined on Cn,d [12, Thm 2.19]. Given a configu- 
ration C, we define a switching in C to be the replacement of two pairs {ci, 62}, 
{63,64} by {61,63}, {62,64} or {61,64}, {63,62}. 

Theorem 4. Let X„ be a random variable defined on Cn,d such that for any 
configurations C, C that differ by a switching 

\X^{C)-X^{C')\<b , 

for some constant b > 0. Then for every t > 0, 

Pr[-^n < E[-^n] ~ < e~ dnb2 and Pr[AT„ > E[X„] -I- t] < e~d^ . 
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Theorem 1 will follow from the following two lemmata. The proof of Lemma 1 
is a straightforward union bound argument and is relegated to the full paper. 

Lemma 1. For any 0 < e < 1/6 and d < n}-l^~^ , w.h.p. every subgraph induced 
by s < vertices contains at most (3/2 — e)s edges. 

Lemma 2. For a given function co(n), let k = k{co,n,p) be the smallest k such 
that 

Pr[x(C„,d) < A:] > l/w(n) . 

With probability greater than 1 — l/w(n), all but S^/ndlog w(n) vertices of Cn,d 
can be properly colored using k colors. 

Proof. For a multigraph G, let Yk{G) be the minimal size of a set of vertices S 
for which G — S is fc-colorable. Clearly, for any k and G, switching two edges of 
G can affect Yi^{G) by at most 4, as a vertex cannot contribute more than itself 
to Yk{G). Thus, if p,k = E[Yfc(C„^d)], Theorem 4 implies 

Pr[Efc < /ife — Av^] < e“TS3 and Pr[Yfc > Mfc + Ai/n] < e“ isa . ( 1 ) 

Define now u = u{n,p,uj{n)) to be the least integer for which Pr[y(G) < u] > 
\/io{n). Choosing A = A(n) so as to satisfy e~^ /(i6d) = l/w(n), the first in- 
equality in (1) yields 

Vr[Yu < Hu - A\/^ < l/uj{n) < Pr[x(G) < m] = Pr[P„ = 0] . 

Clearly, if Pr[Fu < Hu — A\/n] < Pr[F„ = 0] then Hu < \\/n. Thus, the sec- 
ond inequality in (1) implies Pr[F > 2Ai/n] < l/w(n) and, by our choice, 
A = 4: d log Lo{n). □ 



Proof of Theorem 1. The result is trivial for d = 1,2. Given d > 3, let 
k = k{d,n) > 3 be the smallest integer for which the probability that Cn,d 
is /c-colorable is at least 1/ log log n. By Lemma 2, w.h.p. there exists a set of 
vertices S such that all vertices outside S can be colored using k colors and |S'| < 
Sytnd log log log n < Vndlogn = sq. From S, we will construct an increasing 
sequence of sets of vertices {Ui} as follows. Uq = S; for f > 0, Ui+i = UiU 
{wi,W 2 }, where W\,W 2 ^ Ui are adjacent and each of them has some neighbor 
in Ui. The construction ends, with Ut, when no such pair exists. 

Observe that the neighborhood of Ut in the rest of the graph, N{Ut), is always 
an independent set, since otherwise the construction would have gone on. We 
further claim that w.h.p. the graph induced by the vertices in Ut is fc-colorable. 
Thus, using an additional color for N{Ut) yields a (fc -|- l)-coloring of the entire 
multigraph, concluding the proof. 

We will prove that Ut is, in fact, 3-colorable by proving that \Ut\ < sp/e. 
This suffices since by Lemma 1 w.h.p. every subgraph iL of 6 or fewer vertices 
has average degree less than 3 and hence contains a vertex v with deg(w) < 2. 
Repeatedly invoking Lemma 1 yields an ordering of the vertices in H such that 
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each vertex is adjacent to no more than 2 of its successors. Thus, we can start 
with the last vertex in the ordering and proceed backwards; there will always be 
at least one available color for the current vertex. To prove \Ut\ < 2sq log n we 
observe that each pair of vertices entering U “brings in” with it at least 3 new 
edges. Therefore, for every j > 0, Uj has at most sq + 2j vertices and at least 3j 
edges. Thus, by Lemma 1, w.h.p. t < 3so/(4e). □ 



3 Establishing Colorability in Two Moments 



Let us say that a coloring a is nearly-balanced if its color classes differ in size 
by at most 1, and let X be the number of nearly-balanced /c-colorings of C„_d- 
Recall that Ck = 2/clogfc. We will prove that for all k > 3 and d < Ck-i there 
exist constants Ci,C 2 >0 such that for all sufficiently large n (when dn is even), 

/ 1 \ dn/2 

E[X] > ( 1- -j , (2) 

E[X^] < C 2 (^1 - ■ (3) 



By the Cauchy-Schwartz inequality (see e.g. [7, Remark 3.1]), we have Pr[X > 
0] > E[Jf]^/E[X^]> C\/C 2 > 0, and thus Theorem 3. 

To prove (2), (3) we will need to bound certain combinatorial sums up to 
constant factors. To achieve this we will use the following Laplace-type lemma, 
which generalizes a series of lemmas in [2-4] . Its proof is standard but somewhat 
tedious, and is relegated to the full paper. 

Lemma 3. Let £,m be positive integers. Let y € Q™, and let M be a m x £ 
matrix of rank r with integer entries whose top row consists entirely of 1 ’s. Let 
s,t be nonnegative integers, and let ~Vi,Wj G for 1 < i < s and 1 < j < t, 
where each and wj has at least one nonzero component, and where moreover 
J2i=i — X)j'=i '^j- f ■ be a positive twice- differentiable function. 

For n G N, define 

{zGN^:M-z— yn} J-lj — 



and define g : IR^ —>■ IR as 



^(0 = 






m 



where 0° = 1. Now suppose that, conditioned on M ■ f = y, g is maximized at 
some C* with Q > 0 for all i, and write (/max = 5(C*)- Furthermore, suppose 
that the matrix of second derivatives g” = d^g/dfidQ is nonsingular at (*. 

Then there exist constants A,B>0, such that for any sufficiently large n 
for which there exist integer solutions z to M ■ z = yn, we have 
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A < 



,-(^+s-t-r)/2 , 



< B . 



For simplicity, in the proofs of (2) and (3) below we will assume that n is 
a multiple of k, so that nearly-balanced colorings are in fact exactly balanced, 
with n/k vertices in each color class. The calculations for other values of n differ 
by at most a multiplicative constant. 



4 The First Moment 



Clearly, all (exactly) balanced /c-partitions of the n vertices are equally likely to 
be proper fc-colorings. Therefore, E[X] is the number of balanced fc-partitions, 
n!/(n/fc)!^, times the probability that a random d-regular configuration is prop- 
erly colored by a fixed balanced ^-partition. 

To estimate this probability we will label the d copies of each vertex, thus 
giving us {dn — 1)!! distinct configurations, and count the number of such config- 
urations that are properly colored by a fixed balanced fc-partition. To generate 
such a configuration we first determine the number of edges between each pair 
of color classes. Suppose there are hij edges between vertices of colors i and j 
for each i ^ j. Then a properly colored configuration can be generated by i) 
choosing which bij of the dn/k copies in each color class i are matched with 
copies in each color class j ^ i, and then ii) choosing one of the 6^-! matchings 
for each unordered pair i < j. Therefore, the total number of properly colored 
configurations is 



-p-r {dn/k)\ -p-r 

llfr:yi'lPb! = 






i<j 



{dn/k)l’^ 

rii<j - 



Summing over all choices of the {bij} that satisfy the constraints 



Vf : bij = dn/k 



( 4 ) 



we get 



E[X] = 



E 



{dn/k)\^ 



{n/k)\^ {dn - 1)!! n*<j hj'- 

^ ndu /2 n\ {dn/k)\^ ^ {dn/2)\ 

{n/k)l>^ {dn)l ^^lli<jb^J'■ 

By Stirling’s approximation -\/27m (n/e)” < n! < V47rn(n/e)” we get 



TT ^ ■ bii\ 



( 5 ) 



where = 2 d^^ 
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To bound the sum in (5) from below we use Lemma 3. Specifically, z consists 
of the variables bij with i < j, so £ = k{k — 1) /2. For k >3, the k constraints (4) 
are linearly independent, so representing them as M ■ z = yn gives a matrix M 
of rank k. Moreover, they imply = dn/2, so adding a row of I’s to the 

top of M and setting yi = d/2 does not increase its rank. Integer solutions z 
exist whenever n is a multiple of k and dn is even. We set s = 1 and t = £; the 
vector Vi consists of I’s and the Wj are the £ basis vectors. Finally, f{() = 1. 
Thus, £+s — t — r = —{k — 1) and 



9(0 



( d / 2)'*/2 



^ = g(d/2)ff(2C/d) 



where H is the entropy function H{x) = — logXj . 

Since g is convex it is maximized when (/* = d/(2£) for all I < j < £, and g" 
is nonsingular. Thus, gmax = {k{k — l)/2)‘*/^ implying that for some A > 0 and 
all sufficiently large n 



/ 1 \ dn/2 



dnl2 



Setting Cl = Di A completes the the proof. 



5 The Second Moment 

Recall that X is the sum over all balanced fc-partitions of the indicators that 
each partition is a proper coloring. Therefore, E[X^] is the sum over all pairs 
of balanced ^-partitions of the probability that both partitions properly color a 
random d-regular configuration. Given a pair of partitions cr, r, let us say that a 
vertex v is in class (i,j) if o'(w) = i and r(w) = j. Also, let denote the number 
of vertices in each class (i, j). We call A = (aij) the overlap matrix of the pair 
cr, T. Note that since both a and r are balanced 

Vt : Qij = Uji = n/k . (6) 

j j 

We will show that for any fixed pair of fc-partitions, the probability that they 
both properly color a random d-regular configuration depends only on their 
overlap matrix A. Denoting this probability by q{A), since there are n\/Y\ij o-ij- 
pairs of partitions giving rise to A, we have 

= E n4-i '<■"> 

A f f b ■ 

where the sum is over matrices A satisfying (6). 
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Fixing a pair of partitions a and r with overlap matrix A, similarly to the 
first moment, we label the d copies of each vertex thus getting {dn— 1)!! distinct 
configurations. To generate configurations properly colored by both a and r we 
first determine the number of edges between each pair of vertex classes. Let 
us say that there are bijke edges connecting vertices in class (i,j) to vertices in 
class (fc, £). By definition, bijke = bkUj, and if both colorings are proper, bijke = 0 
unless i ^ k and j yf £. Since the configuration is d-regular, we also have 

^ ^ dijke — deiij . ( 8 ) 



To generate a configuration consistent with A and {bijke} we now i) choose 
for each class (i, j), which bijke of its daij copies are to be matched with copies 
in each class (fc,£) with fc yf i and t yf j, and then ii) choose one of the bijke^- 
matchings for each unordered pair of classes i < k, j ^ Thus, 






1 V 


n 


{daij)\ 




Vb 


Yl.k^i,£^j 


odn/2 ^i 3 ^daij)\ 


V 


{dn/2)\ 


{dn)\ 


2-^ 

{ ^ij k£ . 


1 Yli<k,jiie bijke^- 



^ijke^- 

i<k,j^e 



(9) 



where the sum is over the {bijke} satisfying (8). Combining (9) with (7) gives 



E[X^] = E E 

{ciij 



n\ 



Y\ij{daij)\ {dn/2)\ 
{dn)\ n i<k,j^e dijke^- 



(10) 



To bound the sum in (10) from above we use Lemma 3. We let z consist 
of the combined set of variables {uij} U {bijke ■ * < k,j yf £}, in which case 
its dimensionality I (not to be confused with the color £) is + (fc(fc — l))^/2. 
We represent the combined system of constraints (6), (8) as M • z = yn. The 
constraints (8) are, clearly, linearly independent while the 2k constraints (6) 
have rank 2k — 1. Together these imply aij = 1 and j^e ^ijki = d/2, so 
adding a row of I’s to the top of M does not change its rank from r = fc^ + 2A: — 1. 
Integer solutions z exist whenever n is a multiple of k and dn is even. Finally, 
/(C) = 2“^/^, s = k‘^ + 2 and t = k'^ + l + {k{k — l))'^ /2, so £ + s — t — r = — 2(fc— 1). 

Writing aij and Pijke for the components of C corresponding to aij/n and 
bijke /n, respectively, we thus have 



5(C) 



2<i/2_ 



1 ( d / 2)"/2 



n 






d<^ 

doLi 



n 



O^ijk 

i<C.kJ^i ^ijki 






n 



T\ 

U ^3 lU<k,j^l^ijk^ 



( 11 ) 



In the next section we maximize g{C/) over C G satisfying M ■ (/ = y. We 
note that g" is nonsingular at the maximizer we find below, but we relegate the 
proof of this fact to the full paper. 
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6 A Tight Relaxation 



Maximizing g{(^) over C G satisfying M ■ C, = y is greatly complicated by the 
constraints 

J • 'y ^ ~ dcXij . (12) 

To overcome this issue we i) reformulate g{Q and ii) relax the constraints, in a 
manner such that the maximum value remains unchanged while the optimization 
becomes much easier. 

The relaxation amounts to replacing the k'^ constraints (12) with their sum 
divided by 2 , i.e., with the single constraint 

^ = d/2 . (13) 



But attempting to maximize (11) under this single constraint is, in fact, a bad 
idea since the new maximum is much greater. Instead, we maximize the following 
equivalent form of g{(/) 



9(C) 



n 



^ijk£ 

tj ^ij 



n 






n oPijkl 
^ijki 



(14) 



derived by using ( 12 ) to substitute for the exponents daij in the numerator of 
(11). This turns out to be enough to drive the maximizer back to the subspace 

M-C = y. 

Specifically, let us hold {aij} fixed and maximize g{C,) with respect to {/dijki} 
using the method of Lagrange multipliers. Since log 5 is monotonically increasing 
in g, it is convenient to maximize logg instead. If A is the Lagrange multiplier 
corresponding to the constraint (13), we have for all i < k, j ^ 



A = logff(C) = (Ajfe^ log(aijafc^) - (irjkdogldijkt) 

^Hijkt ^Hijkl 

= log aij + log ake ~ log Pijki ~ 1 



and so 

yi < k,j I : Ptjki = Ca^jau, where C = . (15) 

Clearly, such (iijM also satisfy the original constraints (12), and therefore the 
upper bound we obtain from this relaxation is in fact tight. 

To solve for C we sum (15) and use (13), getting 

I ^ ^d^Jkt = ^= = ^~l + J2^l=P ■ 

iakj^i i^k,j^£ ij 

Thus C = d/p and (15) becomes 

daijUke 



Vt < /c,j : Pijki 



P 



(16) 
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Observe that p = p{{aij}) is the probability that a single edge whose end- 
points are chosen uniformly at random is properly colored by both a and t, if 
the overlap matrix is = aijn. Moreover, the values for the bijki are exactly 
what we would obtain, in expectation, if we chose from among the ( 2 ) edges 
with replacement, rejecting those improperly colored by cr or t, until we had 
dn/2 edges - in other words, if our graph model was G{n, m) with replacement, 
rather than Gn,d- 

Substituting the values (16) in (14) and applying (13) yields the following 
upper bound on (/(C): 

, jd/2 TT 

q(C) < b 

1 d‘r‘ 

pdj2 

= 9G(n,m){{oiij}) ■ 

In [4, Thm 5], Achlioptas and Naor showed that for d < Ck-i the function 
9G{n,m) is maximized when = 1/fc^ for all i,j- In this case p = (1 — 1/A:)^, 
implying 

5max < (^1 - 

and, therefore, that for some constant C 2 and sufficiently large n 



E[X^] < 




7 Directions for Further Work 

A Sharp Threshold for Regular Graphs. It has long been conjectured that for 
every k > 2, there exists a critical constant Ck such that a random graph 
G(n, m = cn) is w.h.p. /c-colorable if c < and w.h.p. non-/c-colorable if c > Ck- 
It is reasonable to conjecture that the same is true for random regular graphs, 
i.e. that for all k > 2, there exists a critical integer dk such that a random graph 
Gn,d is w.h.p. fc-colorable ii d < dk and w.h.p. non-fc-colorable if d > dk- If this 
is true, our results imply that for d in “good” intervals (uk,Ck) w.h.p. the chro- 
matic number of Gn,d is precisely fc-l- 1, while for d in “bad” intervals {ck-i,Uk) 
the chromatic number is w.h.p. either fc or fc -|- 1. 

Improving the Second Moment Bound. Our proof establishes that if X, Y are 
the numbers of balanced fc-colorings of Gn,d and G{n,m = dnj2), respectively. 
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then E[X]^/E[X^] = 0(E[y]^/E[F^]). Therefore, any improvement on the up- 
per bound for E[E^] given in [4] would immediately give an improved positive- 
probability fc-colorability result for Gn,d- 

In particular, Moore has conjectured that the function gG{n,m) is maximized 
by matrices with a certain form. If true, this immediately gives an improved 
lower bound, cj, for fc-colorability satisfying — >■ Ufe — 1. This would shrink 
the union of the “bad” intervals to a set of measure 0, with each such interval 
containing precisely one integer d for each k > ko- 

3-Colorability of Random Regular Graphs. It is easy to show that a random 
6-regular graph is w.h.p. non-3-colorable. On the other hand, in [1] the authors 
showed that 4-regular graphs are w.p.p. 3-colorable. Based on considerations 
from statistical physics, Krzsrkala, Pagnani and Weigt [8] have conjectured that 
a random 5-regular graph is w.h.p. 3-colorable. The authors (unpublished) have 
shown that applying the second moment method to the number of balanced 
3-colorings cannot establish this fact (even with positive probability). 
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Abstract. In standard property testing, the task is to distinguish be- 
tween objects that have a property V and those that are e-far from V, 
for some e > 0. In this setting, it is perfectly acceptable for the tester 
to provide a negative answer for every input object that does not satisfy 
V ■ This implies that property testing in and of itself cannot be expected 
to yield any information whatsoever about the distance from the object 
to the property. We address this problem in this paper, restricting our 
attention to monotonicity testing. A function / : {1, . . . ,n} R is at 
distance £/ from being monotone if it can (and must) be modified at 
Efn places to become monotone. For any fixed <5 > 0, we compute, with 
probability at least 2/3, an interval [(1/2 — i5)e,e] that encloses e/. The 
running time of our algorithm is 0(e/^ log log log n), which is optimal 
within a factor of log log and represents a substantial improvement 
over previous work. We give a second algorithm with an expected running 
time of 0(e/^ lognlogloglogn). 



1 Introduction 

Since the emergence of property testing in the nineties [12,8], great progress 
has been made on a long list of combinatorial, algebraic, and geometric testing 
problems; see [11,6,4] for surveys. Property testing is a relaxation of the standard 
decision problem: Given a property V, instead of determining exactly whether 
a given input object satisfies V or not, we require an exact answer only if the 
object satisfies the property or if it is far from doing so. This subsumes a notion 
of distance: Typically the object is said to be £-far from V if at least a fraction e 
of its description must be modified in order to enforce the property. The largest 
such e is called the distance of the object to V. In this setting, the tester can say 
“no” for every input object that does not satisfy V, which precludes the leaking 
of any information regarding the distance of the object to the property. 

This weakness has led Parnas, Ron, and Rubinfeld [10] to introduce the con- 
cept of tolerant property testing. Given 0 < £i < £2 < 1, a tolerant tester must 
accept all inputs that are not £i-far from V and reject all of those that are £ 2 -far 
(and output anything it pleases otherwise). A related problem studied in [10] 

* This work was supported in part by NSF grants CCR-998817, CCR-0306283, ARO 
Grant DAAH04-96-1-0181. 
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is that of estimating the actual distance of the object to the property within 
prescribed error bounds. In the model considered, all algorithms are random- 
ized and err with probability at most 1/3. (or equivalently any arbitrarily small 
constant). 

Testing the monotonicity of functions has been extensively studied [1-3,5, 
7,9]. In the one-dimensional case, given a function / : {l,...,n} i-^- R, after 
querying 0(logn)/£ function values, we can, with probability at least 2/3, ac- 
cept / if it is monotone and reject it if it is £-far from being monotone [3]. 
These methods do not provide for tolerant property testing, however. Very re- 
cently, Parnas, Ron and Rubinfeld [10] designed sublinear algorithms for tolerant 
property testing and distance approximation for two problems: function mono- 
tonicity and clustering. If £/ denotes the distance of / to monotonicity, their 
algorithm computes an estimate £ for £/ that satisfies (l/2)£/ — S<i<Sf + S 
with high probability. The query complexity and running time of their algorithm 
are both 0((logn)^/i5^) (the O notation hides a factor of (log log The 

algorithm maintains and queries a data structure called an “index-value tree.” 
Since the running time is sublinear, the tree is stored implicitly and only relevant 
portions are constructed whenever necessary, using random sampling to make 
approximate queries on the tree. Their construction is sophisticated and highly 
ingenious, but all in all quite involved. 

We propose a simpler, faster, algorithm that is nearly optimal. Given any 
fixed i5 > 0, it outputs an interval [(1/2 — 15)£, £] that encloses £/ with probability 
at least 2/3. The running time is 0{sj^ log log £/^ logn), which is optimal within 
a factor of log log £/^. (The optimality proof is quite simple and omitted from 
this version.) One thing to note is the different use of (5: in our algorithm it is 
part of the multiplicative factor, whereas in [10] it is an additive term. To achieve 
the same multiplicative factor as in our algorithm, the additive term needs to be 
0{Sef). This makes the running time of Parnas et al.’s algorithm 0((log n)^/£^), 
for any fixed S. 

The starting point of our algorithm is the property tester of Ergun et al. [3], 
which relies on a key fact: There exist at least £/n “critical” integers i G 
{l,...,n}; for i to be critical means that it is the (left or right) endpoint of 
an interval at least half of whose elements are in violation with i. Here i is 
said to violate j if either i < j and f{i) > f{j) or i > j and f{i) < /(/). 
By proving an upper bound on the number of critical integers, we are able to 
define a “signature” distribution for / which reflects its distance £/ fairly accu- 
rately. Specifically, two functions with distances to monotonicity off by a factor 
of 2 (roughly) will have signatures that are distinguishable in time 0{ej^ logn). 
This provides us with a tolerant property tester for monotonicity. We can turn 
it into a distance approximator by using a one-way searching strategy, which we 
discuss below. Just as in [10], our algorithm extends to higher dimension. 

We also present an improvement of our one-dimensional algorithm for small 
enough values of £. We show how to estimate £/ in time 0{ej ^ log n log log log n) . 
Unlike in our previous algorithm, the number of steps in this one is itself a 
random variable; therefore, the running time is to be understood in the expected 
sense over the random bits used by the algorithm. 
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2 Estimating Distance to Monotonicity 

Given two functions f,g : R, let d{f,g) = Proh[f{x) ^ g{x)\ 

denote the distance between / and g, where x G {1, . . . , n} is chosen uniformly 
at random. We define Sf = d{f , g) , where At is the set of monotone 

functions from {1, . . . , n} to R. 

Theorem 1. For any fixed 5 > Q, we can compute an interval [(1/2 — 
that encloses £/ with probability at least 2/3. The running time is 
0(e/Mog log £/4og n) . 

It is not entirely clear from the theorem that amplifying the probability of 
success can be achieved by simply repeating the algorithm enough times and 
taking a majority vote. What if we get different candidate intervals every time? 
We do not. As will soon become obvious, majority voting does, indeed, boost 
the probability of success arbitrarily close to 1. 

It is easy to reduce the search for such an interval to a “distance separation” 
decision problem. Suppose that, given any £ > 0, one can tell in 0{e~^logn) 
time and with probability at least 2/3 whether ej > e or £/ < (1/2 — S)e. If 
(1/2 — (j)£ <£/<£, the algorithm can report anything. For each k = 1,2,..., 
we run the algorithm clog(A: + 1) times with £ set to Sk = (1/2 — S)^, where 
c is a large enough constant, and we take a majority vote. We continue until 
we hear the report that £/ > £e- By Chernoff’s bound, the probability that 
££+i ^ Sf < £f_i is at least 1 — X[fe>o > 2/3. The running time of 

Y^i<k<£ ^(log(^+l))^fc ^ logn, which is 0(sy^ log log £/^ logn) time, as claimed. 

This does not quite do the job. Indeed, we are now left with the knowledge 
that £f falls in the interval [£^+i,£^_i], which unfortunately is too big for our 
purposes. It is enclosed in the interval [£o/5,£q], for some 0 < £q < 1, which we 
must now shrink to the right size. To do this we simply use the previous “distance 
separation” algorithm for the values (1 — k6)£o, for 0 < fc < 1/6. This allows us 
to pinpoint £/ within an interval of the form [(1/2 — 0(5) )£, £]. Rescaling 6 gives 
us the desired result. It thus suffices to prove the following lemma: 

Lemma 1. For any fixed £, 5 > 0, we can decide, in time 0(£“^ log n) and with 
probability at least 2/3, whether £f>£ or £f< (1/2 — 5)£. //(1/2 — 5)£ <£/<£, 
the algorithm can report anything. 



2.1 A Separation Oracle 

As mentioned in the introduction, the key to estimating the distance to mono- 
tonicity is to approximate the number of “critical” integers (to be defined in the 
next section). To identify a critical integer z, we need to find an interval starting 
or ending at i such that there are many violations with i in the interval. This 
is done through random sampling, to ensure a sublinear running time. The mo- 
tivation for the following definitions on joint distributions of random variables 
will be made clear later in this section. 
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Let T> be the joint distribution of m independent 0/1 random variables 
xi, . . . , Xm, which can be sampled independently. If Exi < a for all i, then 
V is called a-light; else it is a-heavy. We describe an algorithm light-test which, 
given any a < b, determines whether a distribution is a-light or 6-heavy. 

Lemma 2. If V is either a-light or b-heavy, for some fixed a < b, then with 
probability 2/3 we can tell which is the case in 0{bm/{b — a)^) time. 

Proof. Call light-test({a:i, . . . , Xm}, Co), where cq is chosen so that ci =^co(6 — 
off' jb is a large enough constant. The algorithm runs in time proportional to 
X)fe>o cok{m!2^) = O(com). To see why it works, we begin with a simple obser- 
vation. Suppose that E > 6, then at the fc-th recursive call we sample Xi (if 
at all) exactly Cok times; therefore, by Chernoff’s bounds, 

Prob[x, < (a + 6)/2] = 

The same upper bound holds for the probability that x, > (a -I- 6)/2, assuming 
that E Xi < a. Suppose now that : 

— I? IS 6-heavy: Let Xj be such that Exi >6. At the fc-th recursion call, the 

probability that S' is empty is Summing up over all k bounds the 

likelihood of erring by 1/3. 

— I? IS a- LIGHT: The probability that any given Xi exceeds (a -I- 6)/2 is at 

most 1/3 (conservatively) and so erring any time before the size of S is 
recursively reduced to below ci is X)ci<fc<|S| 2 ~n(k) ^ < 1/6. Af- 

ter that stage, the probability of reaching a 6-heavy verdict is at most 

0(ci(logci)2-^(^i)) < 1/6. 

□ 



light-test (S', k) 

For each x G S, sample it k times and compute the average x; 
Form S' = {x€S|x> (a-|-6)/2}. 

If |S/=0, then output “a-light”. 

If |S/ > |S|/2, then output “b-heavy”. 
light-test (S', k Co) 



2.2 Distance Separation: The Facts 

Given 0 < <5 < 1/2, the integer i is called 5-big if there exists j > i such that 
|{* < k<j\f{k) < /(t)|| > {1/2 - 5){j -i+l) 
or, similarly, j < i such that 

\[j <k <i\f{k) > f{i) 1 1 > (1/2- (5)(i - J -h 1). 
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Intuitively, integer i is big if /(t) violates monotonicity with an abundance of 
witnesses. In the following we show that when b is small, the number of i5-big 
integers approximates efn to within a factor of roughly 2. 

Lemma 3. (i) At least efn integers are 0-big; (ii) no more than (2 + 4i5/(l — 
2i5))e/n integers are S-big. 

Proof. Note that, for any i < j such that f{i) > f{j), either i or j (or both) 
is 0-big. Therefore, if we were to remove all the 0-big integers from the domain 
{1, . . . , n}, the function / would become monotone; hence (i). 

To prove (ii), let C be a set of SfU integers in the domain of / over which 
the function can be modified to become monotone. An element i of C is called 
high-critical (resp. low-critical) if there is j ^ C such that j > i and /(j) < /(z) 
(resp. j < i and /(j) > /(z)). Note that the two definitions are exclusive. For 
each i5-big z, we choose a unique witness ji to its bigness (which one does not 
matter). If ji > i, then z is called right-big] else it is left-big. (Obviously, the 
classification depends on the choice of witnesses.) 

To bound the number of right-bigs, we charge low-criticals with a credit 
scheme. (Then we apply a similar procedure to charge left-bigs.) Initially, each 
element of C is assigned 1 credit. For each right-big i ^ C among n, ... ,1 in 
this order, spread one credit among all the low-criticals k such that i < k < ji 
and f{k) < f{i). We use the word “spread” because we do not simply drop one 
credit into one account. Rather, viewing the accounts as buckets and credits as 
water, we pour one unit of water one infinitesimal drop at a time, always pouring 
the next drop into the least filled bucket. (There are other ways to describe this 
charging scheme, none of them quite as poetic.) 

We now show that no low-critical ever receives an excess of 2 -|- 4i5/(l — 2S) 
credits. Suppose by contradiction that this were the case. Let z be the right- 
big that causes the low-critical k’s account to reach over 2 -|- 4i5/(l — 2S). By 
construction z is not low-critical; therefore, the excess occurs while right-big z is 
charging the I low-criticals k such that i < k < ji and f{k) < f{i). Note that, 
because i ^ C, any k satisfying these two conditions is a low-critical and thus gets 
charged. With the uniform charging scheme (remember the water?), this ensures 
that all of these I low-criticals have the same amount of credits by the time they 
reach the excess value, which gives a total greater than 1(2 -\- 4<5/(l — 25)). By 
definition of right-bigness, I > (1/2 — S)(ji — z -I- 1). But none of these accounts 
could be charged before step jf therefore, 

(1/2 — S)(ji — i -\- 1)(2 -|- 45/(1 — 25)) < ji — i -\- 1, 
which is a contradiction. 

We handle left-bigs in a similar way by running now from left to right, ie, 
z = 1, ... ,n. Since no integer can be both left-critical and right-critical, part (ii) 
of the lemma follows. □ 

2.3 Distance Separation: The Algorithm 

We need one piece of terminology before describing the distance separation al- 
gorithm. Given an interval in [zz,z;], we define two 0/1 random variables a[u,v] 
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and j3[u,v\\ given random i G [u,v] n n}, a[u,v] = 1 (resp. (3[u,v\ = 1) 

iff f{u) > f{i) (resp. f{i) > f{v)). With probability at least 2/3, distance- 
separation {f,s,S). reports that £/ > e (resp. £/ < (1/2— J)£) if it is, indeed the 
case, and anything it wants if (1/2 — 6)e <£/<£. 



distance-separation (/, e, S) 

Pick s = [(1 -f 5/2)e~^ In 2] rEmdom i G {1, . . . , n} . 

For each 1 < fc < (5/(5) Inn, define = a[i, i -|- (1 -|- (5/4)*^] 

and = /?[* - (1 + 5/4)*", *1 • 

Let V be the distribution of . 

If V is (1/2 — (5/4)-heavy , then output “e/ > e” . 

If T> is (1/2 — (5/3)-light , then output “Ef < (1/2 - S)s" . 



The algorithm assumes that both 6 and e/S are suitably small. The require- 
ment on (5 is nonrestrictive. To make £ small, however, we use an artifice: set 
f(i) = -1-00 for i = n -I- 1, . . . , 0{n/6). We also need to assume that the algo- 
rithm used for distinguishing between light and heavy succeeds with probability 
at least 1 — <5^ (instead of 2/3); to do that iterate it log(5“^ times and take a 
majority vote. To prove the correctness of the algorithm, it suffices to show that: 

— If £/ > £, then is (1/2 — 5/4)-heavy with probability 1/2-1- 17((5): 

By Lemma 3 (i), more than en integers are 0-big, so the probability of 
hitting at least one of them in the first step (and hence, of ensuring that T> 
is (l/2)/(l -I- (5/4)-heavy) is at least 1 — (1 — £)* > 1/2 -|- I7(i5). 

— If £/ < (1/2 — (5)£, then V is (1/2 — <5/3)-light with probability 1/2-1- I7(i5): 

By Lemma 3 (ii), the number of (5/3-big integers is less than (1 — S)en; 
therefore, the probability of missing all of them (and hence, of ensuring that 
V is (1/2 — 5/3)-light) is at least (1 — (1 — 5)£)® >1/2-1- 17((5). 

By running the whole algorithm 0{1/S^) times and taking a majority vote, 
we can boost the probability of success to 2/3. By Lemma 2, the running time 
is 0{e~^ logn), as claimed (for fixed 6). This completes the proof of Lemma 1 
and hence of Theorem 1 . 

2.4 A Faster Algorithm for Small Distances 

We show in this section how to slightly improve the query complexity of the 
algorithm to 
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0(min{loglog£^ log log logn} ^ log n). 

The running time is now expected (over the random bits used by the algorithm) . 
To do this, we need the following theorem: 

Theorem 2. We can compute an interval [l7(e/logn),£] that encloses Sf with 
prohahility at least 2/3. The expected running time is 0{eJ^logn). 

Using this theorem, it is clear that the factor log log £/^ in the distance esti- 
mation algorithm can be replaced by minjlog log £/^, log log logn}. Indeed, in- 
stead of taking k = 1,2,3,..., and running the separation oracle for each value of 
£fc anumber oftimes (ie, clog(fc-l-l) times), we redefine Sk to be (1/2— J)^£, where 
£ is the estimate returned by Theorem 2. Because the maximum value of k is now 
0(log log n), the running time drops to 0(min{loglog£)T^, log loglogn}£)T^ log n). 

To prove Theorem 2, we turn to a construction introduced by Goldreich et 
al. [7]. Define a subset P of pairs of integers: (i,j) G P ii j > i, and j — i is at 

most t, where t is the largest power of 2 that divides either i or j. This set has 

the following two properties: 

— |P| = 6>(nlogn). 

— For any i < j, there exists k {i < k < j) such that both (i, k) G P and 
(fc,j) S P. This means, in particular, that for any violation (i,j) of /, there 
exists a “witness” (i,k) or {k,j) of the violation in the subset P. 

Now, for a function /, let M be a maximum matching in the violation graph 
(the undirected graph whose vertex set is {1, . . . ,n| and where i is connected 
to j if i < j and f{i) > /(/)). It is known [7] that \M\ = 0{efn); to be 

precise, ^£/n < \M\ < SfU. Let Q C P be the set of violations of / in P. 

Consider the bipartite graph G with M on the left and Q on the right. Connect 
an edge between (i,j) G M and (a,b) G Q if {i,j} ri {a,b} yf 0. By the second 
property above, and from the definition of a maximum matching, every node 
on the right has degree either 1 or 2, and every node on the left has degree at 
least 1; therefore, the cardinality of the right side is 17(|M|). We would like to 
show that it is 0(|M| logn). If we could do that, then by sampling from P and 
checking for violations, we could then estimate the size of Q and get the desired 
approximation. Unfortunately, it is not quite the case that the cardinality of the 
right side is always 0{\M\ logn). To fix this problem, we need to introduce some 
more randomness. 

We slightly change the definition of P: for an integer r G [1, n] let Pr denote 
the subset of pairs defined as follows: (i,j) G Pr if j — i is at most t, where t is 
the largest power of 2 that divides either z -|- r or j -|- r. The set Pr still has the 
two properties above. In addition, if r is chosen uniformly at random then, for 
any z, the expected number of j such that (z, j) G Pr and j' such that (/', z) G Pr 
is O(logn). The expected number of edges of the corresponding bipartite graph 
Gr, therefore, is 0{\M\logn). So the expected cardinality of the right side is 
a|Pr|, where a G [I2(£//logn), 0(£/)]. We sample Pr to form an estimation 
a for a and return e = Galogn, for some large enough constant C, to prove 




236 



Nir Ailon et al. 



Theorem 2. The estimation follows the predictable scheme: (1) pick a random 
r G n}; (2) pick a pair (i,j) uniformly at random from (3) if (*, j) is 

a violation of /, output success, otherwise failure. The success probability is 
precisely a, so repeating the sampling enough times sharpens our estimation to 
the desired accuracy, as indicated by the following fact. 

Lemma 4. Given a 0/1 random variable with expectation a > 0, with probabil- 
ity at least 2/3, the value of 1/a can be approximated with a relative constant 
error by sampling it 0(1/ a) times on average. Therefore, a can be approximated 
within the same error and the same expected running time. 

Proof. Run Bernoulli trials on the random variable and define Y to be the num- 
ber of trials until (and including) the first 1. It is a geometric random variable 
with E y = 1/a, and var (T) = (1 — a) /a^ < (E F)^. By taking several samples 
of y and averaging we get an estimate 1/a of 1/a. Using Chebyshev’s inequality, 
a constant number of samples suffices to get a constant factor approximation. 

□ 
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Abstract. Consider the following communication problem, that leads 
to a new notion of edge coloring. The communication network is repre- 
sented by a bipartite multigraph, where the nodes on one side are the 
transmitters and the nodes on the other side are the receivers. The edges 
correspond to messages, and every edge e is associated with an integer 
c(e), corresponding to the time it takes the message to reach its destina- 
tion. A proper fc-edge-coloring with delays is a function / from the edges 
to {0, l,...,fc — 1}, such that for every two edges ei and 62 with the same 
transmitter, /(ei) yf /(e2), and for every two edges ei and 62 with the 
same receiver, f{ei)+c{ei) ^ /(e2) -l-c(e2) {mod k). Maxell, Wilfong and 
Winkler [10] conjectured that there always exists a proper edge coloring 
with delays using k — A + 1 colors, where A is the maximum degree of 
the graph. We prove that the conjecture asymptotically holds for simple 
bipartite graphs, using a probabilistic approach, and further show that 
it holds for some multigraphs, applying algebraic tools. The probabilistic 
proof provides an efficient algorithm for the corresponding algorithmic 
problem, whereas the algebraic method does not. 



1 Introduction 

Motivated by the study of optical networks, Maxell, Wilfong and Winkler con- 
sidered in [10] a communication network in which there are two groups of nodes: 
transmitters and receivers. Each transmitter has to send a set of messages, each 
of which should reach one receiver (more than one message per receiver is al- 
lowed). Each message has an associated delay, which is the time from the moment 
it is sent until it reaches its destination. The network is timed by a clock, so all 
times are integers. We wish to find a periodic setup of message sending for all 
transmitters, such that in each cycle all messages of all transmitters are sent, 
where each transmitter sends at most one message and each receiver gets at 

* Research supported in part by a USA-Israeli BSE grant, by the Israel Science Foun- 
dation and by the Hermann Minkowski Minerva Center for Geometry at Tel Aviv 
University. 
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most one message, at any given time unit. The objective is to find such a cycle 
of minimal length. 

We can formalize this problem as follows: we represent the network by a 
bipartite multigraph G = {V, E) with sides A and B, where the vertices of A are 
the transmitters and the vertices of B are the receivers. The edges correspond to 
the messages that are to be sent, and a function c : E ^ N associates each edge 
with its delay. The aim is to find the smallest number k for which there exists 
an edge coloring / : E {0,1, ...,fc — 1}, such that for every two edges ci and 
62 that have a common vertex in A, /(ei) yf /(c 2 ), and for every two edges ei 
and 62 that have a common vertex in B, /(ci) + c( 6 i) ^ /(c 2 ) + 0 ( 62 ) {mod k). 

The minimum number of colors is, clearly, at least A, where A is the maxi- 
mum degree in G. Furthermore, there are simple examples in which Z\ -I- 1 colors 
are required, for example, any Z\-regular graph in which all delays but one are 
0. Haxell, Wilfong and Winkler [10] raised the following conjecture. 

Conjecture 1. Let G = (V, E) be a bipartite multigraph with sides A and B and 
with maximum degree A, and let c : if ^ fV be a delay function. Then there is 
a coloring f : E ^ {0, 1, . . . , A}, such that for every two edges ei and 62 that 
have a common vertex in A, /(ei) yf /(e 2 ), and for every two edges 61 and 62 
that have a common vertex in B, /(ei) -I- c(ei) ^ f{^ 2 ) + c(c 2 ) {mod {A + 1)). 

In this paper, we show that the conjecture asymptotically holds for simple 
graphs. More precisely, we prove that if G is a simple bipartite graph with 
maximum degree A, then A + o{A) colors suffice. Our proof uses a random 
coloring procedure, following the result of Kahn [12] and its extensions and 
variants by Haggkvist and Janssen [11] and Molloy and Reed [14,15] on list 
edge coloring. 

Using algebraic techniques, based on [2, 3, 7], we also show that the conjecture 
holds for some families of bipartite multigraphs such as all even length regular 
multi-cycles where the degree of regularity plus one is a prime. We further de- 
scribe a generalization of the problem to non-bipartite multigraphs, and prove 
the conjecture in some cases. 

In section 2 we outline the proof of the asymptotic result for simple bipar- 
tite graphs, in section 3 we present the algebraic proofs for some families of 
multigraphs. The final section 4 contains some concluding remarks. 

2 Simple Bipartite Graphs 

In this section we show that if G is a simple bipartite graph with maximum 
degree A, then there is a coloring with the required properties using (l-|-o(l))Z\ 
colors. 

Theorem 1. For every e > 0 there is a Aq = L\o(e) such that the following 
holds for every A > Aq: 

Let G = (y, E) be a bipartite simple graph with sides A and B and with maximum 
degree A, and let c : E ^ N be a delay function. Then for k = [(1-1- e)Z\] there 
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is a coloring f : E ^ {0, 1, . . . ,k — 1}, such that for every two edges ei and C 2 
that have a common vertex in A, /(ei) ^ /(e 2 ), and for every two edges ei and 
62 that have a common vertex in B, /(ei) + c(ei) ^ /(e 2 ) + c(e 2 ) {mod k). 

Throughout the section we omit all floor and ceiling signs, whenever these 
are not crucial. We use, throughout the paper, the notation 0{x) to denote, as 
usual, 0 (x(logx)‘^(^)). 

Let G = (V, E) be a simple bipartite graph with sides A and B, and assume 
without loss of generality that G is Z\-regular. Let c : E ^ N he the delay 
function, and let fc = (1 + e)A, where e > 0 is an arbitrarily small constant. 
We show that if A is sufficiently large, then there is a coloring with the desired 
properties using k colors. 

We present a coloring procedure with three stages: 

1. Choose a small set of reserved colors for every edge. 

2. Iteratively color the edges as follows. In each iteration, assign every uncolored 

edge a random color from the unreserved colors that are still available for it. 

An edge retains the color assigned to it only if no adjacent edge is assigned 

a conflicting color. 

3. Complete the coloring from the lists of reserved colors. 

In the rest of this section we describe some of the details of each stage, and 
prove that, with positive probability, the procedure finds a coloring with the 
required properties. Due to space limitations, most of the technical details are 
postponed to the full version of the paper. 



2.1 Notation 

To simplify the description of the procedure, we extend the function c by defining 
for all e = {u,v) € E with u G A and v G B, c(e,u) = 0 and c(e,v) = c(e). 
Now the coloring / should satisfy the condition that for every two edges ei, 62 G 
E with a common vertex u, /(ei) + c(ei,u) yf f{^ 2 ) + c(e 2 ,u), where all the 
operations on the colors are done modulo k. 

During the iterative procedure, we denote by Le C {0, 1, . . . , fc — 1} the set 
of all unreserved colors still available for e, for all e G E. For all v G C let Ly = 
{ c + c{{u,v),v) I cG (u,v) G E }. For a set of colors C and a color c, let 

C-c= { c'-c I c' G C }. Clearly, C {Lu-c{{u,v),u))n{Ly-c{{u,v),v)), 

for all {u,v) G E. Thus, every time a color c is removed from a list then 
c — c(e, v) is removed from the lists Lg, for all edges e incident with v. 

For all 6 G C and c G Ly, denote by the set of all uncolored edges e 
incident with v, for which the color c — c(e, v) is still available, i.e. 

Ty^c = { e = {u,v) G E \ c — c(e, v) G Ly and e is uncolored }. 

In the first stage we choose a set of reserved colors for every edge. We do it by 
choosing a set of reserved colors ReservCy for all v GV , and then by defining for 
every edge e = {u,v) G E, Reserve^ = {Reserveu — c{e,u))r\{Reservey — c{e,v)). 
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During the iterative procedure, for dll v & V and color c, we denote by Rv,c 
the set of all uncolored edges e = (u,u) incident with v, such that the color 
c — c(e, v) + c(e, u) is reserved for u, i.e. 

Rv,c = { e = {u,v) & E \ c—c(e, v) + c(e, u) € ReservCu and {u, v) is uncolored} . 

For all i, e G E, V G V and color c, let ll, 1}, t} ,, and r* ^ denote the sizes of 
the sets Lg, ^ and Rv,c, respectively, after i iterations. If after i iterations 

c ^ Ly we define t\, g = 0. For all the above, i = 0 refers to the time before the 
first iteration, but after the first stage of the procedure, in which the reserved 
colors are chosen. 



2.2 Coloring Scheme Overview 

We now describe the coloring procedure. Start by choosing a small set of reserved 
colors ReservCy for all v G V, and by defining for all e = (u, v) G E, ReservCg = 
{ReservCu — c(e,u)) n {ReservCy — c(e,v)). We would like these colors to be 
available for e at the final stage, and hence we remove from the initial set Eg of 
every edge e = (u, v) G E, all the colors that conflict with the colors reserved 
for u and v. Therefore, for all e = {u, v) G E, the initial set Eg is defined by 
Le = {0, 1, . . . , fc — 1} \ {{ReservCu — c(e, u)) U {ReservCy — c(e, u))). We choose 
the reserved colors so that for all e G E, l^> {1 + f )^, but we also make sure 
that for all e G E, |i?eseruee| > and for all u G F and color c, r^ g < |Z\, 

so the coloring can be completed at the final stage. 

The second stage consists of a constant t = t{e) number of iterations. In each 
iteration, we assign to each edge e G E a color c chosen randomly from Eg. If 
there is a conflict, i.e. if there is an adjacent edge e' to which we assigned a 
color c' such that c + c(e, v) = c' + c(e', v), where v is the common vertex of e 
and e', we uncolor e. If an edge e = (u, v) retains the color c it was assigned, 
then adjacent edges cannot be colored with conflicting colors. Thus we remove 
c + c(e,u) from L„, c+ c(e,v) from L„, and the corresponding colors from the 
lists Eg/ of the edges e' incident with u and v. 

We start with > (1 + §)A, for all e G E, t^ g < A and r^ g < |Z\ for all 
V G V and color c. We show that the sizes of the sets Eg and Ty^g decrease at 
roughly the same rate, and hence, for every i, the sizes t} g are slightly smaller 
than the sizes 1],. 

The f-th iteration is carried out as follows. We first remove colors from the 
sets Eg so that they all have the same size, which we denote by U-i- Now, for 
all e = (u,v) G E assign e a color c chosen randomly from Eg. The color c is 
removed from e if an adjacent edge is assigned a conflicting color, i.e. if there is 
an edge ei incident with u that is assigned c+ c(e,u) — c(ei,u), or an edge 62 
incident with v that is assigned c + c(e,v) — c(e2,v). Therefore, the probability 
that e retains c is 



( 1 -— ) 
H-1 



,,c+c(e,u) 



+ K 



,c+c(e,v) 



-2 
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For every color c G Ly, the edges e incident with v that may retain c— c(e, v) 
are the edges in For every e G Ty^c, the probability that e is assigned the 
color c and retains it is roughly j— There is at most one edge e G that 
retains c— c(e, v). Therefore, these events are disjoint, and if one of them occurs 
we remove c from Ly. Hence, the probability that we remove c from Ly is roughly 



For an edge e = (u, v) and a color c G Lg, we remove c from Lg if we remove 
c + c(e, u) from or c + c(e, v) from Ly. If these events were independent, the 
probability that we do not remove c from L^, would have been roughly (1 — 

We later show that although there is some dependence between those events, the 
probability that we do not remove c from Le is still close to (1 — Hence, 

For every v € V and c G Ly, if e = (u, v) G Ty^c, we remove e from Ty^c if e 
retains the color it is assigned, or if e cannot be colored with c— c(e, v) anymore, 
since c — c(e,v) + c(e,u) was removed from If these events were independent, 
the probability that e remains in Ty^c would have been roughly (1 — since 

the probability of each of these events is about e“^. We show later that the 
dependence between the above events does not change this probability much. 
Therefore, 

As for the sets Ry^c, we remove an edge e from Ry^c only if it retains the 
color it was assigned. Therefore, 

EKJ « (1 - 

We execute the iterative procedure for t iterations, where we choose t so that 

2 

with positive probability r* ^ < gfgQ^ for all u G H and color c. We can then 
prove, using the local lemma (see, e.g., [4] chapter 5, or the results in [16, 17, 9]), 
that in this case the coloring can be completed with the reserved colors. 

Due to space limitations, the detailed proofs of the first and last stages are 
omitted, and will appear in the full vesions. In the next subsection we describe 
some of the details of the main part of our coloring scheme, the iterative proce- 
dure. 



2.3 The Iterative Procedure 

In each iteration, we assign every edge e a color c chosen randomly from Lg. We 
let e retain c only if it causes no conflicts, i.e. no adjacent edge e' is assigned 
c-|-c(e, u) — c(e', u), where u is the common vertex of e and e! . If an edge e = (u, v) 
retains a color c, we remove c-|-c(e, u) from L„ and c-|-c(e, v) from Ly. To simplify 
the proof, we remove some colors from some of the lists Lg at the beginning of 
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each iteration, so that all the lists have the same size, and we sometimes remove 
colors from edges and from the lists even when there is no conflict, in order 
to equalize all the probabilities. The details are explained below. 

We show that the size of each and of each decreases by a factor of 
approximately (1 — e~^)^ in each iteration, and the size of each by a factor 
of approximately 1 — e“^. Each Ty^c is initially slightly smaller than each Ly, 
and since these sizes decrease at approximately the same rate, the above holds 
at the beginning of every iteration. 

Consider the z-th iteration, and assume > k-i for all e G E, and < 
li-i for all f G V and color c. We first remove colors from some of the lists Lg so 
that they all have size exactly h-i. For each edge e = {u, v) and color c G Lg, the 
probability that no adjacent edge is assigned a conflicting color, if e is assigned 
c, i.e. that no edge e\ incident with u is assigned c+ c{e,u) — c(ei,u), and that 
no edge 62 incident with v is assigned c + c(e, v) — c(c 2 , v), is 



P(e,c) 




t* i r N+t* \ , n-2 

u,c-\-c{e,u) v,c-i-c(e,v) 



> 




> e 



-2 



We now use an equalizing coin flip, so that if e is assigned c and no adjacent 
edge is assigned a conflicting color, we still remove c from e with probability 

This ensures that the probability that e retains c, conditional of e receiving c, is 
precisely 

P(e,c)(l - Eq{e,c)) = e"^. 

For every v G V and c G Ly, there is at most one edge e G Ty^g that retains 
c — c(e,v). Therefore, the events that e retains c — c(e,v) for all e G Ty g are 
pairwise disjoint, and hence the probability that we do not have to remove c 
from Ly, i.e. the probability that no edge e G Ty^g retains c — c(e, v) is 



Q{v, c) = 1 — , > 1 — e ^ 



li—\C 



2 — 



If we do not have to remove c from Ly, we still remove it with probability 

l-e-2 



Vq{v,c) = 1 - 



Q{v,c) 

This ensures that the probability that c remains in Ly is precisely 
Q{v,c){l - Vq{v,c)) = 1 - e"^. 



Lemma 1. For every v G V and color c, ^] = (1 — e g . 

Proof. An edge is removed from Ry^g if and only if it retains the color it is 
assigned. This happens with probability e“^. Therefore, E[rl^g] = (1 — 

’ □ 
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Lemma 2. For each e G E, E[ll] > (1 — 

Lemma 3. For every v G V and color c G Ly, < (1 ~ + e“^. 

The proofs of Lemmas 2 and 3 require a more careful analysis, and will appear 
in the full version. 

In Lemmas 1, 2 and 3, we have shown that the expected size of every Lg 
and Ty^c decreases in one iteration by a factor of approximately (1 — and 

of every Ry^c by a factor of approximately 1 — e~^. We wish to prove that with 
positive probability, the sizes of all these sets indeed decrease by approximately 
these factors. For this purpose we show that these values are highly concentrated. 
More precisely, we prove that there exists a constant /3 > 0, such that the 
following hold for all 1 < i < t. 



Lemma 4. For each e G E, 



Pr 




E[ll]\ >logZ\v^ 






where the probability is over the random choices of the i-th iteration. 
Lemma 5. For each v GV and cG Ly, if tl~f > then 



Pr 



K,c~E[tlJ\ > log A^/l~^ 



< g-/31og^ ^ 



where the probability is over the random choices of the i-th iteration. 



1 ^ 

Lemma 6. For each v gV and color c, if > gfogLl then 



where the probability is over the random choices of the i-th iteration. 

The proofs of Lemmas 4, 5 and 6 use Talagrand’s inequality (see, e.g. [4, 15]), 
and will appear in the full version. 

To complete the proof we apply the local lemma to each iteration, to show 
that with positive probability, the size of every set is within a small error of the 
expected size. 

Lemma 7. With positive probability, the following hold after i iterations, for all 
t)<i<t: 

1. For all e G E 

ll>{l-e-^r (l+l)A-d{^). 

2. For all v G V and color c 



Pr 



yy,c~E[rlj\ >logZ\i 



tl^y<{l-e-y^A + d{VA). 
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3. For all V € V and color c 

Proof. We prove the lemma by induction on i. Suppose the claim holds after 
i — 1 iterations. Note that by the induction hypothesis, < k-i. Thus, we 
can apply all the lemmas that were proved in this section. For all v G V and 
color c, if < (1 — or riff < (1 — then the corresponding 

condition after i iterations trivially holds. Otherwise, we can apply Lemmas 5 
and 6. 

For all e £ E, let Ag be the event that II violates the first condition, and 
for all V £ V and color c, let and be the events that tl g and r* g 
violate the second and third conditions, respectively. By Lemmas 4, 5 and 6, the 
probability of each one of these events is less than for some constant 

[3. Every event Ag is mutually independent of all other events except for those 
that correspond to the vertices incident with e and their neighbours, and to 
edges incident with these vertices. Thus, it is mutually independent of all but 
0(Z\^) of the other events. Every event By^g or C„,c is mutually independent of 
all other events except for those that correspond to the edges incident with v, to 
the vertices incident with them and to the neighbours of these vertices. Thus, it 
is mutually independent of all but 0{A^) of the other events. Thus, by the local 
lemma, all the conditions hold with positive probability. 

To complete the proof of the lemma we use the bounds on If, tf g and rf g 
given by the first stage of our procedure. □ 



3 Algebraic Methods 

In this section we prove Conjecture 1 for several families of graphs using algebraic 
methods. The relevance of results from additive number theory to this conjecture 
appears already in [10], where the authors apply the main result of Hall [8] to 
prove their conjecture for the multigraph with two vertices and d parallel edges 
between them. Here we consider several more complicated cases. 

3.1 Even Multi- cycles 

Consider the case where the graph G is a d-regular multigraph whose underlying 
simple graph is a simple cycle of even length. We show that if d + 1 is a prime 
then there exists a coloring with the required properties using d + 1 colors. 

In our proof, we use the following theorem proved in [3]: 

Theorem 2. (Combinatorial Nullstellensatz) 

Let F he an arbitrary field, and let P = P{x\, . . . ,Xn) he a polynomial in 
F[x\, ...,Xn]- Suppose that the degree of P is where each U is a non- 

negative integer, and suppose the coefficient ofYlf^i in P is nonzero. Then, 
if Si, ..., Sn are subsets of F with jS'^j > fi, there are si G Si, S 2 £ S 2 , ..., Sn £ Sn 
so that P{si, ..., Sn) 0. 
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Let G = (V, E) be a simple even cycle with, possibly, multiple edges. Let A 
and E be the sides of G, and denote the vertices of A by oi, 02, • ■ • , a„, and the 
vertices of E by 61, 62, • ■ • , &n, such that there are edges between Oj and bi for 
all 1 < i < n, between bi and for all 1 < z < n — 1, and between and 
a\. Since G is d regular, all the edges {ai,bi) have the same multiplicity s, and 
hence all the other edges have multiplicity t = d — s. 

We now associate a polynomial P with the graph G. For every edge e, we 
have a variable Xg- For every two edges ei and 62 which have a common vertex 
in A we have a term — Xg^), and for every two edges ei and 62 which have 
a common vertex in E we have a term {xg,^ + c(ei) — Xg^ — 0(62)) ■ Thus, the 
polynomial P is defined by 

P= (Xei-XeJ (Xei + c(ei) - CCea - C(e2)) 

eine2nAi^il eine2nBi^$ 

Since the graph is d-regular, every edge has d — 1 terms for each one of its 
vertices. Hence, the total degree of P is nd{d — 1). 

Proposition 1. // d + 1 is a prime then the coefficient of the monomial 
rieGE P nonzcro modulo d + 1. 

Proof. The monomial Oees is of maximum degree, and thus its coefficient 
in P is equal to its coefficient in the polynomial 

Q= (Xei-XeJ (Xej-CCeJ 

eir\e2nAi^li eine2nBiilD 

In [7] (see also [2]) it is shown that, for any d-regular planar multigraph, the ab- 
solute value of this coefficient is equal to the number of proper d-edge-colorings. 
Every d-edge-coloring of G is obtained by partitioning the colors into a subset 
of size s and a subset of size t, and for every two connected vertices, choosing 
a permutation of the appropriate set. Thus the number of edge colorings with 
d colors is = d!(s!)"“^(f!)"“^, which is nonzero modulo d-l- 1 since 

d -|- 1 is a prime. □ 

Corollary 1. Let G = (V,E) be an even length d-regular multi-cycle, where 
d -I- 1 is a prime. Let (A,E ) be a bipartition of G, and let c : E ^ N be a delay 
function. Then, there is a coloring f : E ^ {0, 1 , . . . , d} such that if ei and 62 
have a common vertex in A then /(ei) yf f{e-2), and if e\ and 62 have a common 
vertex in E then /(ei) -I- c(ei) ^ /(e2) -f 0(62) {mod (d -I- 1)). 

Proof. By Theorem 2, with Sg = {0,1 ,..., d} and te = d — 1 for every e G E, 
there is a function f : E ^ {0, 1 , . . . , d} such that the value of P, when every 
Xg is assigned the value /(e), is nonzero modulo d -I- 1. Thus, for every two 
edges ei and 62 that have a common vertex in A, /(ei) yf /(c2) since we have a 
term (xgj^ — Xg^) in P, and for every two edges ei and 62 that have a common 
vertex in E, /(ei) -|- c(ei) ^ /(e2) -f 0(62) (mod (d -I- 1)) since we have a term 
(ccei -I- c(ei) - - 0(62)) in P. □ 
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3.2 Multi-ii:4 

The problem can be generalized to non-bipartite multigraphs. In this case, we 
specify for every edge which endpoint is the transmitter and which is the receiver. 
Following the notations used in section 2, for every edge e = (u,v), where u is 
the transmitter and v is the receiver, we define c{e,u) = 0, and c(e,v) to be the 
delay associated with e. 

We can further generalize the problem as follows. Given a multigraph G = 
(V,E), where every edge e = (u,v) G E is associated with two integers c{e,u) 
and c(e,u), we aim to find the smallest number k such that there is a coloring 
/ : E —>■ {0,1,. ..,/c — 1}, satisfying the following. For every vertex v and every 
two edges ei yf C 2 incident with v, /(ei) + c{ei,v) ^ /(e 2 ) + c{e 2 ,v) {mod k). 
In this case, A+1 colors do not always suffice, simply because x'(G) may be as 
large as where X^(G) is the edge chromatic number of G. The following 

conjecture seems plausible for this case. 

Conjecture 2. Let G = {V,E) be a multigraph, and suppose that every edge 
e = {u,v) G E is associated with two integers c{e,u) and c(e,v). Let k = 
x'{G) + 1- Then there exists a coloring f : E ^ {0,1, ■■■, k — 1} such that for 
every vertex v and every two edges ei yf C 2 incident with v, /(ei) + c{ei,v) ^ 
/(e 2 ) + c(c 2 , f) {mod k). 

Proposition 2 . Let G = {V,E) he a d-regular multi-K^, where every edge e = 
{u,v) G E is associated with two integers c{e,u) and c{e,v), and suppose d+ 1 is 
a prime. Then there is a coloring f : E ^ {0, 1, . . . , d}, satisfying the condition 
that for every vertex v and every two edges e\ yf 62 incident with v, f{ei) + 
c{ei,v) ^ /(e 2 ) + c(e 2 , v) {mod {d + 1 )). 

The proof is similar to the proof of the result for even cycles. 

Proof. First, note that if G is d-regular, then every pair of non-adjacent edges 
of K 4 have the same multiplicity in G. Denote these multiplicities by a, b and c. 
We associate the following polynomial with G: 

P= {Xei + c{ei,v) - Xe 2 - c{e 2 ,v)). 

vGV e-i^e^^E : v^e\r\e 2 

P is a polynomial in 2d variables, with total degree 2d{d— 1). Hence, by Theorem 
2, if the coefficient of {lees is nonzero modulo d+ 1, then there is a function 
/ : P— > {0,1,. ..,d}, such that if every variable Xe is assigned /(e) then the value 
of P is nonzero modulo d -I- 1, and therefore / satisfies the required properties. 
The coefficient of OeeB i^ ^ is equal to its coefficient in 

Q (^ei ^ 62 ); 

vGV ei^e 2 ^E : v^e\C\e 2 



which is, by [7], the number of proper d-edge-colorings of G. Such a coloring 
is obtained as follows. First, choose a permutation of the colors and color the 
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edges incident to a vertex u accordingly. Now for every pair of other vertices v 
and w, the set of colors that may be used to color the edges between v and w 
is the same set used for the edges between u and the forth vertex, and we only 
choose the permutation of this set. Thus, the number of proper d-edge-colorings 
is d\a\b\c\ ^ 0 {mod {d + 1)), since d + 1 is a prime. □ 

4 Concluding Remarks 

In section 3 we proved Conjecture 1 for some graphs using algebraic techniques. 
This method can be used to prove the conjecture for several other graphs. How- 
ever, it seems that in order to prove the conjecture for the general case, more 
ideas are needed. In the graphs for which we used Theorem 2, the theorem im- 
plies that there is a proper edge coloring with delays using Z\ -|- 1 colors, even if 
there is one forbidden color for every edge. 

In section 2 we showed that Conjecture I asymptotically holds for simple 
bipartite graphs. It would be interesting to extend this proof to multigraphs as 
well. 

In the probabilistic proof presented in section 2, we proved that A + o{A) 
colors suffice, and did not make any effort to minimize the o{A) term. By mod- 
ifying our proof slightly, we can show that A + 0{A^^^) colors suffice, and it 
seems plausible that a more careful analysis, following the discussion in [14], can 
even imply that A + 0(\/A) colors suffice. 

The probabilistic proof can be extended to other variations of edge coloring. 
Instead of the delays, one can associate with every edge e and an endpoint v of 
e an injective function on the colors. Then, by our proof, there is an edge 
coloring / using A + o{A) colors such that for every vertex v and any two edges 
ei yf 62 incident with v, 9 ei,v{f{ei)) yf 9 e 2 ,v{f{e 2 ))- 

The known results about the algorithmic version of the local lemma, initiated 
by Beck ([5], see also [1],[13], [6]), can be combined with our probabilistic proof in 
Section 2 to design a polynomial time algorithm that solves the corresponding 
algorithmic problem. In contrast, the algebraic proofs of Section 3 supply no 
efficient procedures for the corresponding problems, and it will be interesting to 
find such algorithms. 

Acknowledgement 

We would like to thank B. Sudakov and P. Winkler for fruitful discussions. 

References 

1. N. Alon, A Parallel algorithmic version of the Local Lemma, Random Structures 
and Algorithms, 2 (1991), 367-378. 

2. N. Alon, Restricted colorings of graphs, in ’’Surveys in Combinatorics”, Proc. 14th 
British Combinatorial Conference, London Mathematical Society Lecture Notes 
Series 187, edited by K. Walker, Cambridge University Press, 1993, 1-33. 




248 



Noga Alon and Vera Asodi 



3. N. Alon, Combinatorial Nullstellensatz, Combinatorics, Probability and Comput- 
ing, 8 (1999), 7-29. 

4. N. Alon and J. H. Spencer, The Probabilistic Method, Second Edition, Wiley, 2000. 

5. J. Beck, An algorithmic approach to the Lovasz Local Lemma, Random Structures 
and Algorithms, 2 (1991), 343-365. 

6. A. Czumaj and C. Scheideler, Coloring Nonuniform Hypergraphs: A New Algo- 
rithmic Approach to the General Lovasz Local Lemma, Random Structures and 
Algorithms, 17 (2000), 213-237. 

7. M. N. Ellingham and L. Goddyn, List edge colourings of some 1-factorable multi- 
graphs, Gombinatorica, 16 (1996), 343-352. 

8. M. Hall, A combinatorial problem on abelian groups, Proc. Amer. Math. Soc., 3 
(1952), 584-587. 

9. P. E. Haxell, A note on vertex list colouring. Combinatorics, Probability and Com- 
puting, 10 (2001), 345-348. 

10. P. E. Haxell, G. T. Wilfong and P. Winkler, Delay coloring and optical networks, 
to appear. 

11. R. Haggkvist and J. Janssen, New bounds on the list chromatic index of the com- 
plete graph and other simple graphs, Gombinatorics, Probability and Computing, 
6 (1997), 273-295. 

12. J. Kahn, Asymptotically good list-colorings, Journal of Combinatorial Theory Se- 
ries A, 73 (1996), 1-59. 

13. M. Molloy and B. Reed, Further Algorithmic Aspects of the Local Lemma, Pro- 
ceedings of the 30th Annual ACM Symposium on Theory of Computing, May 1998, 
524-529. 

14. M. Molloy and B. Reed, Near-optimal list colorings. Random Structures and Al- 
gorithms, 17 (2000), 376-402. 

15. M. Molloy and B. Reed, Graph Golouring and the Probabilistic Method, Springer 

( 2001 ). 

16. B. Reed, The list colouring constants, Journal of Graph Theory, 31 (1999), 149-153. 

17. B. Reed and B. Sudakov, Asymptotically the list colouring constants are 1, Journal 
of Combinatorial Theory Series B, 86 (2002), 27-37. 




Small Pseudo-random Families of Matrices: 
Derandomizing Approximate Quantum Encryption 



Andris Ambainis^ and Adam Smith^’* 

^ Institute for Advanced Study, Princeton, NJ, USA 
ambainisoias . edu 

^ MIT Computer Science and AI Lab, Cambridge, MA, USA 
asmithOcsail .mit . edu 



Abstract. A quantum encryption scheme (also called private quantum channel, 
or state randomization protocol) is a one-time pad for quantum messages. If two 
parties share a classical random string, one of them can transmit a quantum state 
to the other so that an eavesdropper gets little or no information about the state 
being transmitted. Perfect encryption schemes leak no information at all about 
the message. Approximate encryption schemes leak a non-zero (though small) 
amount of information but require a shorter shared random key. Approximate 
schemes with short keys have been shown to have a number of applications in 
quantum cryptography and information theory [8]. 

This paper provides the first deterministic, polynomial-time constructions of 
quantum approximate encryption schemes with short keys. Previous construc- 
tions [8] are probabilistic - that is, they show that if the operators used for en- 
cryption are chosen at random, then with high probability the resulting protocol 
will be a secure encryption scheme. Moreover, the resulting protocol descriptions 
are exponentially long. Our protocols use keys of the same length as the proba- 
bilistic constructions; to encrypt n qubits approximately, one needs n + o(n) bits 
of shared key [8], whereas 2n bits of key are necessary for perfect encryption [3]. 
An additional contribution of this paper is a connection between classical combi- 
natorial derandomization and constructions of pseudo-random matrix families in 
a continuous space. 



1 Introduction 

A quantum encryption scheme (or private quantum channel, or state randomization 
protocol) allows Alice, holding a classical key', to scramble a quantum state and send 
it to Bob (via a quantum channel) so that (1) Bob, given the key, can recover Alice’s state 
exactly and (2) an adversary Eve who intercepts the ciphertext learns nothing about the 
message, as long as she doesn’t know the key. We do not assume any shared quantum 
states between Alice and Bob, nor any back channels from Bob to Alice^. 

* A. A. supported by NSF grant DMS-01 1 1298. A.S. supported by a Microsoft Ph.D. Fellowship. 

* Classical keys are inherently easier to store, distribute and manipulate, since they can be 
copied. More subtly, encryption with a shared quantum key is in many ways a dual problem to 
encryption with a classical key; see [8, 5] for more discussion. 

^ A back channel from Bob to Alice would allow using quantum key distribution to generate a 
long secret key. However, such interaction is often impossible, e.g. if Alice wants to encrypt 
stored data for her own later use. 

K. Jansen et al. (Eds.): APPROX and RANDOM 2004, LNCS 3122, pp. 249-260, 2004. 
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There are two variants of this definition. An encryption scheme is called perfect 
if Eve learns zero information from the ciphertext, and approximate if Eve can learn 
some non-zero amount of information. A perfect encryption ensures that the distribu- 
tions (density matrices) of ciphertexts corresponding to different messages are exactly 
identical, while an approximate scheme only requires that they be very close; we give 
formal definitions further below. In the classical case, both perfect and approximate en- 
cryption require keys of roughly the same length - n bits of key for n bits of message. 
In the quantum case, the situation is different. 

For perfect encryption, Ambainis et al. [3] showed that 2n bits of key are necessary 
and sufficient to encrypt n qubits. The construction consists of applying two classical 
one-time pads - one in the “standard” basis {|0), |1)} and another in the “diagonal” 
basis {^(|0) + |1)),^(|0)-|1))}. 

Approximate encryption was studied by Hayden, Leung, Shor and Winters [8]. They 
introduced an additional, useful relaxation: they showed that if the plaintext is not en- 
tangled with Eve’s system to begin with, then one can get approximate quantum en- 
cryption using only n + o(n) bits of key - roughly half as many as are necessary for 
perfect encryption^. The assumption that Eve’s system is unentangled with the message 
is necessary for this result; otherwise roughly 2n bits are needed, even for approximate 
encryption. The assumption holds in the quantum counterpart of the one-time pad situ- 
ation (one party prepares a quantum message and sends it to the second party, using the 
encryption scheme) as long as the message is not part of a larger cryptographic protocol. 

Hayden et al. [8] showed that a random set of unitary matrices leads to a 

good encryption scheme with high probability (to encrypt, Alice uses the key to choose 
one of the matrices from the set and applies the corresponding operator to her input). 
However, verifying that a particular set of matrices yields a good encryption scheme 
is not efficient; even writing down the list of matrices is prohibitive, since there are 
exponentially many of them. 

This paper presents the first polynomial time constructions of approximate quantum 
encryption schemes (to relish the oxymoron: derandomized randomization protocols). 
The constructions run in time O(n^) when the message p consists of n qubits. That is, 
given the key and the input message, Alice can produce the output using steps 

on a quantum computer. The key length we achieve is slightly better than that of the 
probabilistic construction of [8]. Our results apply to the trace norm on matrices; exact 
results are stated further below. 

The main tools in our construction are small-bias sets [10] of strings in {0, 1}^". 
Such sets have proved useful in derandomizing algorithms [10], constructing short 
PCPs [6] and the encryption of high-entropy messages [12]. Thus, one of the contri- 
butions of this paper is a connection between classical combinatorial derandomization 
and constructions of pseudo-random matrix families in a continuous space. Specifically, 
we connect! Fourier analysis over to Fourier analysis over the matrices . 

This parallels to some extent the connection between quantum error-correcting codes 
over n qubits and classical codes over GF(4)”. 

^ The result of [8] highlights an error in the proof of a lower bound on key length of authentica- 
tion schemes in [4]. The results of that paper remain essentially correct, but the definition of 
authentication requires some strengthening, and the proof of the lower bound is more involved. 
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Definitions. We assume that the reader is familiar with the basic notation of quantum 
computing (see [11] for an introduction). Syntactically, an approximate quantum en- 
cryption scheme is a set of 2^ invertible operators \k € {0, 1}^}. The Ek’s may be 
unitary, but need not be: it is sufficient that one be able to recover the input p from the 
output Ek,{p), which may live in a larger-dimensional space than p. Each takes n 
qubits as input and produces n' > n qubits of output. If n' = n then each operator E^, 
corresponds to a unitary matrix [7^, that is Ei^{p) = Uk,pU\. 

For an input density matrix"^ p, the density matrix of the ciphertext from the adver- 
sary’s point of view is: 

£(p) = E. [e;„(p)] = 1 ^ E,{p) 

Ke{o,i}'= 

When the scheme is length-preserving, this yields £(p) = ^ U^pU}.. 

Definition 1. The set of operators {E,f\ is an approximate quantum encryption scheme 
with leakage e (also called “e-randomizing scheme") for n qubits if 



for all density matrices p on n qubits: 



D(£{p),^l) 



£{p) 




tr 



< £• 

( 1 ) 



Here I refers to the identity matrix in dimension 2" , and refers to the trace 

distance between density matrices. The trace norm of a matrix cr is the trace of the 
absolute value of a (equivalently, the sum of the absolute values of the eigenvalues). 
The trace distance between two matrices p, a is the trace norm of their difference: 



cr) = Up - a\\^^ = Tr(|p - cr|) 



This distance plays the same role for quantum states that statistical difference plays 
for probability distributions: the maximum probability of distinguishing between two 
quantum states p, a via a single measurement is ^ + jD(p,a). One can also measure 
leakage with respect to other norms; see below. 



Remark 1. This dehnition of quantum encryption implicitly assumes that the message 
state p is not entangled with the adversary’s system. Without that assumption the def- 
inition above is not sufficient, and it is not possible to get secure quantum encryption 
using n(l + o(l)) bits of key (roughly 2n bits are provably necessary^). Thus, this sort 
of construction is not universally applicable, and must be used with care. 



Previous Work. Ambainis et al. [3] considered perfect encryption; this corresponds to 
the case where e = 0. The choice of matrix norm is irrelevant there, since 5(p) = ^I. 
As mentioned above, they showed that 2n bits of key are necessary and sufficient. The 

Recall that for a pure state \f), the density matrix p is \(j)){(j)\. 

^ This folklore result appears more or less explicitly in both [4, 8]. Similar arguments show that 
n bits of key are necessary to encrypt n classical bits, even with access to quantum computers 
(but not interaction). 
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construction uses the key to choose one of 2^" Pauli operators (defined below) and 
applies that to the input state. 

Hayden et al. [8] showed that a set of 0(n2”/e^) unitary operators suffices. They 
showed this both for the trace norm, and for the “operator norm,” discussed below. For 
the trace norm, they also showed that a random set of Pauli matrices of the same size 
would suffice. This means that for encrypting n qubits, they gave a non-polynomial- 
time, randomized scheme requiring n + log n -I- 2 log(l/e) -I- 0(1) bits of key. 



Our Results. We present three explicit, polynomial time constructions of approximate 
state randomization protocols for the trace norm. All are based on existing constructions 
of i5-biased sets [10,2, 1], or on families of sets with small average bias. The three 
constructions are explained and proven secure in Sections 3.1, 3.2 and 3.3, respectively. 

The first construction is length-preserving, and requires n -I- 2 log n -I- 2 log(l/e) -I- 
0(1) bits of key, roughly matching the performance of the non-explicit construction. 
The second construction is length-increasing: it encodes n qubits into n qubits and 2n 
classical bits but uses a shorter key: only n -I- 2 log(l/ e) bits of key are required. Both 
of these constructions are quite simple, and are proven secure using the same Fourier- 
analytic technique. 

The final construction has a more sophisticated proof, but allows for a length- 
preserving scheme with slightly better dependence on the number of qubits: 

n -I- min {21ogn -b 21og(l/e), logn -b 31og(l/e)} -b 0(1) 
bits of key. The right-hand term provides a better bound when e > ^ ■ 



Randomization Schemes for Other Norms? Definition 1 measures leakage with re- 
spect to the trace norm on density matrices, This is good enough for encryption 
since the trace norm captures distinguishability of states. However, Hayden et al. [8] 
also considered randomization schemes which give guarantees with respect to a differ- 
ent norm, the operator norm. 

A guarantee on the operator norm implies a guarantee for the trace norm, but 
schemes with the operator norm guarantee also have a host of less cryptographic appli- 
cations, for example: constructing efficient quantum data hiding schemes in the LOCC 
(local operation and classical communication) model; exhibiting “locked” classical cor- 
relations in quantum states [8]; relaxed authentication of quantum states using few bits 
of key [9]; and transmitting quantum states over a classical channel using n + o(n) bits 
of communication, rather than the usual 2n bits required for quantum teleportation [5]. 

More formally, for a dxd Hermitian matrix A with eigenvalues {Ai,A 2 ,...,Ad}, the 
operator norm (or oo-norm) is the largest eigenvalue, ||A|j^ = max |Ai|, theFrobenius 
norm is the Euclidean length of the vector of eigenvalues, || AH 2 = (X)i )^^^> ^nd the 
trace norm is the sum of the absolute values of the eignvalues, 1^*1- 

easy to see the chain of inequalities: 



\\A\\^,<Vd\\A\\,<d\\A\\^. 
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We can then state the condition for a map E to be e-randomizing map for n qubits 
in three forms of increasing strength. For all input states p on n qubits: 

||f (p) - < e; ||£(p) - < e/V 2 ^; ||£(p) - ^l||^ < e/ 2 ". 

Our constructions satisfy the definition with respect to the Frobenius norm, but they 
are not known to satisfy the stronger operator-norm definition. This suggests two inter- 
esting questions. First, is it possible to prove that the other applications of state random- 
ization schemes require only a guarantee on the Frobenius norm? Second, is it possible 
to design explicit (i.e. polynomial-time, deterministic) randomization schemes that give 
good guarantees with respect to the operator norm? 

The remainder of this paper describes our constructions and their proofs of security. 



2 Preliminaries 



Small-Bias Spaces. The bias of a random variable ^ in {0, 1}" with respect to a string 
a G {0, 1}" is the distance from uniform of the bit aQ>A, where © refers to the standard 
dot product on Z 2 : 

A{a) = [(- 1 )“®^] = 2 Pr[a © A = 0 ] - 1 . 

The function A is the Fourier transform of the probability mass function of the distri- 
bution, taken over the group Z 2 . 

The bias of a set S' G {0, 1}” with respect to a is simply the bias of the uniform 
distribution over that set. A set S is called i5-biased if the absolute value of its bias is at 
most S for all a ^ 0". 

Small-bias sets of size polynomial in n and l/i5 were first constructed by Naor and 
Naor [10]. Alon, Bruck et al. (ABNNR, [1]) gave explicit (i.e. deterministic, polynomial- 
time) constructions of (5-biased sets in {0, 1}” with size 0(n/5^). Constructions with 
size 0(n^/(5^) were provided by Alon, Goldreich, et al. (AGHP, [2]). The AGHP con- 
struction is better when 8 = o(l/n). In both cases, the string in a set can be con- 
structed in roughly v? time (regardless of S). 

One can sample a random point from a i5-biased space over {0, 1}” using either 
logn + 31og(l/(5) + 0(1) bits of randomness (using ABNNR) or using 21ogn + 
21og(l/(5) bits (using AGHP). 



Small-Bias Set Families. One can generalize small bias to families of sets (or random 
variables) by requiring that on average, the bias of a random set from the family with 
respect to every a is low [7]. Specifically, the expectation of the squared bias must be at 
most 6^. Many results on (5-biased sets also hold for i5-biased families, which are easier 
to construct. 



Definition 2. A family of random variables (or sets) {Ai\^^^ is 5-biased if 







< 6^ for all a 7 ^ 0 ". 
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Note that this is not equivalent, in general, to requiring that the expected bias he less 
than 6. There are two important special cases: 



1 . 

2 . 



If S' is a i5-biased set, then {S} is a (5-biased set family with a single member; 

A family of linear spaces is S-biased if no particular word is contained in 

the dual C:^ of a random space Q from the family with high probability. Specih- 
cally: 



a (a) 



0 if a ^ 

1 if a G 



Hence a family of codes is (5-biased if and only if Pri^/[a G C:^] < <5^, for every 
Of ^ 0”. Note that to meet the dehnition, for linear codes the expected bias must be 
at most 6^, while for a single set the bias need only be 6. 



One can get a good 5-biased family simply by taking {Q} to be the set of all linear 
spaces of dimension k. The probability that any fixed non-zero vector a lies in the dual 
of a random space is exactly 6^ = , which is at most 2“^. 

One can save some randomness in the choice of the space using a standard pairwise 
independence construction. View {0, 1}" as GF(2"), and let K C Gf(2") be an 
additive subgroup of size 2^. For every non-zero string a, let the space Ca be given 
by all multiples an, where k € K, and the product is taken in GF{7^). The family 
{Ca I a G GF(2”), 0^0} has the same bias as the set of all linear spaces (S < 2“^/^). 
To see this, let {ki, ..., Kk} be a basis of K (over GF{2)). A string a is in G^ if and 
only if a © (aKi) = ••• = «© (aKk) = 0. This is a system of k linearly independent 
constraints on a, and so it is satished with probability = 2“^ when a ^ GF{2^), 
and even lower probability when we restrict a to be non-zero. Choosing a set Ca from 
the family requires n bits of randomness. 



Entropy of Quantum States. As with classical distributions, there are several ways to 
measure the entropy of a quantum density matrix. WeTl use the analogue of collision 
entropy (a.k.a. Renyi entropy). 

For a classical random variable A on {0, 1}”, the collision probability of two inde- 
pendent samples of X ispc = The Renyi entropy of A is — logpc- 

For a quantum density matrix p, the analogous quantity is — log Tr(p^). If the eigen- 
values of p are {px}, then the eigenvalues of are {p^}, and so Tr(p^) is exactly the 
collision probability of the distribution obtained by measuring p in a basis of eigenvec- 
tors. i/Tr(p2) is called the Frobenius norm of p. 

If p is the completely mixed state in d dimensions, p = then Tr(p^) is 1 /d. The 
following fact states that any other density matrix for which this quantity is small must 
be very close to I. The fact follows by applying the (standard) inequality Tr(|Z\|)^ < 
(iTr(Z\^) to the Hermitian matrix A = p — I/d. 

Fact 1. Ifp is d-dimensional quantum state and Tr(p^) < I?(p, ^I) < e. 

Pauli Matrices. The 2x2 Pauli matrices are generated by the matrices: 
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The Pauli matrices are the four matrices {][,X,Z,XZ}. These form a basis for 
the space of all 2 x 2 complex matrices. Since XZ = —ZX, and Z^ = X^ = 1, 
the set generated by X and Z is given by the Pauli matrices and their opposites: 
{±l±X,±Z,±XZ}. 

If u and V are n-bit strings, we denote the corresponding tensor product of Pauli 
matrices by X'^Z'" . That is, if we write u = (ul, m„) and v = (ui, u„), then 

(The strings x and z indicate in which positions of the tensor product X and Z 
appear, respectively.) The set {X^Zy \ u,v G {0, 1}”} forms a basis for the 2” x 2” 
complex matrices. The main facts we will need are given below: 

1 . Products of Pauli matrices obey the group structure of {0, 1}^” up to a minus sign. 
That is, (X^Z^){X^Z^) = 

2. Any pair of Pauli matrices either commutes or anti-commutes. Specifically, 

3. The trace of X'^Z'’ is 0 if {u, v) ^ 0^" (and otherwise it is Tr(I) = 2"). 

4. {X^Z'^y = 

Pauli Matrices and Fourier Analysis. The Pauli matrices form a basis for the set of all 
2" X 2" matrices. Given a density matrix p, we can write p = a„ 

This basis is orthonormal with respect to the inner product given by -^Tr{A^ B), where 
A, B are square matrices. That is, ^Tr((X“Z*')^A“Z^) = 5a,u5b,v 

Thus, the usual arithmetic of orthogonal bases (and Fourier analysis) applies. One 
can immediately deduce certain properties of the coefficients au,v in the decomposi- 
tion of a matrix p. First, we have the formula au,v = ^Tr(Z’'A“p). Second, the 
squared norm of p is given by the squared norm of the coefficients, that is ^Tr(pip) = 
V IttiijtiP- Since p is a density matrix, it is Hermitian (pi = p). One can use this 
fact, and the formula for the coefficients a„ to get a compact formula for the Renyi 
entropy (or Frobenius norm) in terms of the decomposition in the Pauli basis: 

U,V 

3 State Randomization and Approximate Encryption 

3.1 Encrypting with a Small-Bias Space 

The ideal quantum one-time pad applies a random Pauli matrix to the input [3]. Con- 
sider instead a scheme which first chooses a 2n-bit string from some set with small bias 
6 (we will set 5 later to be e2“”/^). If the set of strings is B we have: 

f(po) = ^ E 

' ' (a,b)eB 
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That is, we choose the key from the set B, which consists of 2n-bit strings. To encrypt, 
we view a 2n-hit string as the concatenation (o, b) of two strings of n bits, and apply 
the corresponding Pauli matrix. 

(The intuition comes from the proof that Cayley graphs based on e-biased spaces 
are good expanders: applying a Pauli operator chosen from a i5-biased family of strings 
to po will cause all the Fourier coefficients of po to be reduced by a factor of <5, which 
implies that the “collision probability” (Frobenius norm) of po also gets multiplied by 
5. We expand on this intuition below. ) 

As a first step, we can try to see if a measurement given by a Pauli matrix 
can distinguish the resulting ciphertext from a totally mixed state. More explicitly, we 
perform a measurement which projects the ciphertext onto one of the two eigenspaces 
of the matrix X^Z^. We output the corresponding eigenvalue. (All Pauli matrices have 
two eigenvalues with eigenspaces of equal dimension. The eigenvalues are always either 
— 1 and 1 or —i and i.) 

To see how well a particular Pauli matrix X'^Z'" will do at distinguishing, it is 
sufficient to compute 

|Tr(X“Z’'£:(po))|. 

This is exactly the statistical difference between the Pauli measurement’s outcome 
and a uniform random choice from the two eigenvalues. We can compute it explicitly: 

Tr(X“Z"£:(po)) = Tr(A“Z"E(,,i,)es [X<^ Z^ p^Z^ X^]) 

= Ea.b [Tr(A“Z’'A“zVo^'’^“)] 

= Ea.b [Tr(Z*'A“A:“Z’’A:“Z*'po)] 

= Ea.b [(-l)“0'’+'>0“] Tr(X“Z>o) 

Since a©u + 6©ttis linear in the concatenated 2n-bit vector (a, 6), we can take 
advantage of the small bias of the set B to get a bound: 

|Tr(X“Z’'£(po)) I < ,5|Tr(A“Z"po) | when (u, u) ^ 0^” 

Equivalently: if we express po in the basis of matrices then each coefficient 

shrinks by a factor of at least <5 after encryption. We can now bound the distance from 
the identity by computing Tr(£(po)^): 

Tr(£(po)2) = |Tr(A“Z"£(po))|2 

U,V 

+ F ^ |Tr(A“Z>o)P<^(l + 5"2"Tr(p2)) 

Setting S = e2“"/^, we get approximate encryption for all states (since Tr(pQ) < 1). 
Using the constructions of AGHP [2] for small-bias spaces, we get a polynomial-time 
scheme that uses n + 2 log n + 2 log(l / e) bits of key. 
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3.2 A Scheme with Shorter Key Length 

We can improve the key length of the previous scheme using (5-biased/am^7^c^ of sets. 
The tradeoff is that the resulting states are longer: the ciphertext consists of n quhits and 
2n classical hits. In classical terms, the encryption algorithm uses additional random- 
ness which is not part of the shared key; in the quantum computing model, however, 
that randomness is “free” if one is allowed to discard ancilla quhits. 

Lemma 1. a family of subsets o/{0, 1}^" with average square bias S^, 

then the operator 

£{po) = Eiex [|i)(i| 0 EafeeA. 

is an approximate encryption scheme for n qubits with leakage e whenever 5 < 

Before proving the lemma, we give an example using the small-hias set family from 
the preliminaries. View the key set {0, 1}^ as an additive subgroup K of the field F = 
For every element a € F, define the set Ca = {o«:|k G K}. The family 
{Ca} has bias 5 < 2“^/^ (Section 2). The corresponding encryption scheme takes a 
key K e {0,1}'= C GF(22"): 



£{po]k) 



'Choose a GF(22") \ {0} 

Compute the product aK G GF(2^") 

Write aK as a concatenation (a, b), where a,b G (0, 1}” 

Output the classical string a and the quantum state pqZ^ X'^ 



With a quantum computer, random bits are not really necessary for choosing a; it is 
sufficient to prepare 2n EPR pairs and discard one qubit from each pair. For the scheme 
to be secure, the bias <5 should be less than a/ e/2”, and so the key only needs to be 
n + 2 log(l/ e) bits long. The main disadvantage is that the length of the ciphertext has 
increased by 2n classical bits. 



Proof As before, the proof will use elementary Fourier analysis over the hypercube 
Z 2 ", and intuition comes from the proof that Cayley graphs based on e-biased set fam- 
ilies are also expanders. 

Think of the output of the encryption scheme as a single quantum state consisting 
of two systems: the first system is a classical string describing which member of the 
(5-biased family will be used. The second system is the encrypted quantum state. To 
complete the proof, it is enough to bound the collision entropy of the entire system by 
^(1 + ^^)- 

For each i G X (that is, for each member of the set family), let pi denote the encryp- 
tion of po with a random operator from the set Ai . The first step of the proof is to show 
that the collision entropy of the entire system is equal to the average collision entropy 
of the states pi . 

Claim. Tr{£{pof) = [Tr(p-)] 
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Proof. We can write £(/to) = Yli I*) (*l ® Pi- Then we have 
Tr(£(po)^) = ]^EijTr((|i)(i||j)(j|) 

Since (i\\j) = Sij, we get Tr(£:(po)^) = ]^ Zi Tr(/9^), as desired. □ 

Take any string w = {u, v) G {0, 1}^", where u,v G {0, 1}”. Recall that Ai{u, v) 
is the ordinary Fourier coefficient (over Z^") of the uniform distribution on Ai, that is 
Ai(u, v) = Ea^Ai [(—1)“®™]. From the previous proof, we know that 

Tr(W“ZV,) = Mv,u) ■ Tr(X“Z>o). 

We can now compute the average collision entropy of the states pi. Using linearity 
of expectations: 

E4Tr(p2)]=E, ^ + ^ Y. |Tr(X“Z»P 

(u,u)/0 

= ^ + ^ Y E4|Tr(X“Z»P] 

(u,u)/0 

= ^ + 2 ^ E E, [Ai{v,uf] |Tr(W“Z>o)P 

{u,v)^Q 

The expression E^ Ai {v,u)'^ is exactly the quantity bounded by the (squared) bias . 
As in the previous proof, the entropy Tr(£(po)^) is bounded by + <^^2”Tr(/jQ)). 

By our choice of <5, the entropy is at most (1 + e^), and so £{pq) is within trace 
distance e of the completely mixed state. □ 

3.3 Hybrid Construction 

Let d be a prime between 2” and 2”+^. Then, it suffices to show how to randomize a 
state in a c?-dimensional space Hd spanned by|i),i G — 1}, since a state 

on n qubits can be embedded into Hd. We define X and Z on this space by X\j) = 
|(j + 1) mod d) and Z\j) = Notice that X^ Z^ = and 

= Z~^X~f (The definitions of X and Z are different than in the previous 
sections, since we are operating on a space of prime dimension). 

We start with a construction that uses n + 1 bits of randomness and achieves ap- 
proximate encryption for e = 1. (Notice that this is a non-trivial security guarantee. The 
trace distance between perfectly distinguishable states is 2. Distance 1 means that the 
state cannot be distinguished from ^ with success probability more than 3/4.) We will 
then extend it to any e > 0, using more randomness. 

Let 

- d— 1 

s(p) = 

d . 
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Claim. 

rr(£(p)2)< i(l + rr(p2)). 

Proof. Let p' = £[p). 

ij i 



The first sum is equal to d-^ = ^ because pP = ^ X)fc=i Pkk = ^. To calculate the 
second sum, we split it into sums St = Pi i+tiPi i+t)* t = 1, 2, . . . , d — 1. (In 
the indices for pij and pC, we use i + f as a shortcut for {i + t) mod d.) We have 



d-l 

Pi,i+t ~ j ^ ^ tn Pi—a,i—a+t^ 

a—Q 



where w is the root of unity. 

^d-l 



“ ^2 ( ^ V \pi+a,i+t+a\ + ^ ^ ^ Pi-a,i-ht-a(pi-b,'i-l-t-b) 

a,b,a^b 



i a—0 



Therefore, 



where 



1 ■ ° ' 1 

‘5't = ■^ ^ k^i,jPi,i+t{Pj,j+t) 



1=1 ip=j 



(.. . — yjiP+af -U+af)t — yja*2(i-j)t 






Since d is a prime, 2(t — j)t is not divisible by d. Therefore, X)a ~ 

Cy =0,St = 2 \Pi,i+t\'^ and 



Tr{{pT) = ^ + 2Y\P^j\^- 



□ 



ByFactl,f9(£(p),i)<l. 

We now improve this construction to any e. Let B be an e-biased set on to = [log d] 
bits. For b G {0, 1}™, define a unitary transformation Ub as follows. Identify numbers 
0, 1, . . . , d — 1 with strings x G {0, 1}"*. Dehne t7h|x) = (— l)*'®®|x), with & ©x being 
the usual (bitwise) inner product of b and x. (Note that Ub is just to the Z operator over 
a different group. It is the same Z operator used in the previous sections). Let 

£'{p) = Y UbpUl and£"(p) = £{£'{p)). 

bGB 
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We claim that £" is e-approximate encryption scheme. W.l.o.g., assume that p is a pure 
state |t/>) = Ci|*)- Then = ac*. Let p' = J2bGB UbpUl be the result of 

encrypting p by Then, 

' ' h^B ' ' h^B 

Since B is e-biased, \p^y\ < APxy\ for x,y, x / y. Therefore, \p'xy\ < 

e'Bx^y \Pxy\- Together with the Claim above and Fact 1, this implies that S” is e- 
randomizing. The number of key bits used by 8" is n + log \ B\ + 0(1) which is n + 
2 log n + 2 log i + 0(1) if the AGHP scheme is used and n + log n + 3 log ^ + 0(1) 
if ABNNR is used. The first bound is the same as the one achieved by using small-bias 
spaces directly (Section 3.1). The second bound gives a better result when e > T. 
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Abstract. We address the problems of pattern matching and approxi- 
mate pattern matching in the sketching model. We show that it is im- 
possible to compress the text into a small sketch and use only the sketch 
to decide whether a given pattern occurs in the text. We also prove a 
sketch size lower bound for approximate pattern matching, and show it 
is tight up to a logarithmic factor. 



1 Introduction 

Pattern matching is the problem of locating a given (smaller) pattern in a (larger) 
text. It is one of the most fundamental problems studied in computer science, 
having a wide range of uses in text processing, information retrieval, computa- 
tional biology, compilers, and web search. These application areas typically deal 
with large amounts of data and therefore necessitate highly efficient algorithms 
in terms of time and space. 

In order to save space, I/O, and bandwidth, large text files are frequently 
stored in compressed form. The naive method for locating patterns in compressed 
files is to first decompress the files, and then run one of the standard pattern 
matching algorithms on them. Amir and Benson [2] initiated the study of pat- 
tern matching in compressed files; their approach is to process the compressed 
text directly, without first decompressing it. Their algorithm, as well as all the 
subsequent work in this area [3, 21, 12, 24, 11, 22, 15], deal with lossless compres- 
sion schemes, such as Huffman coding and the Lempel-Ziv algorithm. The main 
focus of these results is the speedup gained by processing the compressed text 
directly. 

In this paper we investigate a closely related question: how succinctly can 
one compress a text file into a small “sketch”, and yet allow locating patterns 
in the text using the sketch alone? In this context we consider not only lossless 
compression schemes but also lossy ones. In turn, we permit pattern matching 
algorithms that are randomized and can make errors with some small constant 
probability. Our main focus is not on the speed of the pattern matching al- 
gorithms but rather on the succinctness of the compression. Highly succinct 
compression schemes of this sort could be very appealing in domains where the 
text is a massive data set or when the text needs to be sent over a network. 

A fundamental and well known model that addresses problems of this kind 
is the sketching model [8,14], which is a powerful paradigm in the context of 
computations over massive data sets. Given a function, the idea is to produce 
a fingerprint {sketch) of the data that is succinct yet rich enough to let one 
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compute or approximate the function on the data. The parameters that play a 
key role in the applications are the size of the sketch, the time needed to produce 
the sketch and the time required to compute the function given the sketch. 

Results. Our first main result is an impossibility theorem showing that in the 
worst-case, no sketching algorithm can compress the text by more than a con- 
stant factor and yet allow exact pattern matching. Specifically, any sketching 
algorithm that compresses any text of length n into a sketch of size s and en- 
ables determining from the sketch alone whether an input pattern of length 
m = l7(logn) matches the text or not with a constant probability of error re- 
quires s > f2{n — to). We further show that the bound is tight, up to constant 
factors. 

The proof of this lower bound turns out to be more intricate than one might 
expect. One of the peculiarities of the problem is that it exhibits completely 
different behaviors for to < (1 — o(l))logn and to > logn. In the former case, 
a simple compression of the text into a sketch of size 2™ is possible. We prove 
a matching lower bound for this range of to as well. These results are described 
in Section 3. 

Our second main result is a lower bound on the size of sketches for approxi- 
mate pattern matching, which is a relaxed version of pattern matching: (i) if the 
pattern occurs in the text, the output should be “a match”; (ii) if every substring 
of the text is at Hamming distance at least k from the pattern, the output should 
be “no match”. An arbitrary answer is allowed if neither of the two holds. We 
prove that any sketching algorithm for approximate pattern matching, requires 
sketch size fl{nlm), where n is the length of the text, to is the length of the pat- 
tern, and the Hamming distance at question is fc = em, for a fixed 0 < e: < 1. We 
further show that this bound is tight, up to a logarithmic factor. These results 
are described in Section 4. 

Interestingly, Batu et al. [6] showed a sampling procedure that solves (a 
restricted version of) approximate pattern matching using 0{njm) non-adaptive 
samples from the text. In particular, their algorithm yields a sketching algorithm 
with sketch size 0{n/m). This procedure was the main building block in their 
sub- linear time algorithm for weakly approximating the edit distance. The fact 
that our sketching lower bound nearly matches their sampling upper bound 
suggests that it might be hard to improve their edit distance algorithm, even in 
the sketching model. 

Techniques. A sketching algorithm naturally corresponds to the communication 
complexity of a one-way protocol. Alice holds the text and Bob holds the pattern. 
Alice needs to send a single message to Bob (the “sketch”), and Bob needs to 
use this message as well as his input to determine whether there is a match or 
not^. 

^ Usually, a sketching algorithm corresponds to the commnnication complexity of a 
simnltaneons messages protocol, which is equivalent to summarizing each of the text 
and the pattern into a small sketch. However, in the context of pattern matching, it 
is reasonable to have a weaker requirement, namely, that only the text needs to be 
summarized. 
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The most classical problem which is hard for one-way communication com- 
plexity is the indexing function: Alice is given a string x G {0,1}” and Bob is 
given an index i G {1, ... ,n}, and based on a single message from Alice, Bob has 
to output Xi. It is well known that in any protocol solving this problem, even 
a randomized one, Alice’s message has to be of length I7(n). Our lower bound 
for approximate pattern matching is proved by a reduction from the indexing 
function. 

Our lower bound for exact pattern matching uses a reduction from a variant 
of the indexing function. In this variant, Alice gets a string x G (0, 1}”; Bob gets 
an index i G [n] and also the m — 1 bits preceding x^ in x; the goal is to output 
Xi. Using tools from information theory we prove an fi(n — m) lower bound for 
this problem in the one-way communication complexity model. 

Related Work. Pattern matching and approximate pattern matching have a rich 
history and extensive literature - see, for instance, the excellent resource page 
[20]. To the best of our knowledge, pattern matching, has not been considered in 
the sketching model. For approximate pattern matching, the only relevant result 
appears to be the above mentioned work of Batu et al. [6]. 

Sketching algorithms for various problems, such as estimation of similarity 
between documents [8,7,9], approximation of Hamming distance [19,13] and 
edit distance [4] between strings, and computation of Lp distances between vec- 
tors [1, 14], have been proposed in the last few years. Sketching is also a useful 
tool for approximate nearest-neighbor schemes [19, 16], and it is related to low- 
dimensional embeddings and to locality-sensitive hash functions [16]. 

2 Preliminaries 

2.1 Communication Complexity 

A sketching algorithm is best viewed as a public-coin one-way communication 
complexity protocol. Two players, Alice and Bob, would like to jointly compute 
a two-argument function f : X x y ^ Z. Alice is given x G X and Bob is 
given y G y . Based on her input and based on randomness that is shared with 
Bob, Alice prepares a “sketch” sa{x) and sends it to Bob. Bob uses the sketch, 
his own input y, and the shared randomness to determine the value f{x,y). 
For every input (x, y) G X x Y, the protocol is required to be correct with 
probability at least 1 — i5, where 0 < (5 < 1 is some small constant. Typically, 
the error probability <5 can be reduced by repeating the procedure several times 
independently (in parallel). 

The main measure of cost of a sketching algorithm is the length of the sketch 
Sy 4 (x) on the worst-case choice of shared randomness and of the input x. Another 
important resource is the amount of randomness between Alice and Bob. New- 
man [23] shows that the amount of shared randomness can always be reduced 
to 0(log 1^) at the cost of increasing the protocol’s error probability by S'. In 
one-way protocols, Alice can privately generate these 0(log ji) and send them 
to Bob along with the sketch SA(a:). 
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Some of our lower bounds use a reduction from the standard indexing prob- 
lem, which we denote by INDj: Alice is given a string x S {0, 1}*, Bob is 
given j G [t], and the goal is to output xj. This problem has a lower bound 
of t(l — H 2 {S)) in the one-way communication complexity model [18,5]. 

2.2 Pattern Matching and Approximate Pattern Matching 

For a Boolean string x G {0, 1}” and integer 1 < j < n, let Xi denote the jth 
bit in X. For integers 1 < i < j < n, [i,j] denotes the corresponding integer 
interval, [n] the interval [1, n] = {1, . . . , n}, and x[i,j] denotes the substring of x 
that starts at position i and ends at position j. We define the pattern matching 
and approximate pattern matching problems in the communication model. 

Let 0 < TO < n. In the (n, to) pattern matching problem, denoted 
Alice gets a string x G {0,1}” and Bob gets a string y G {0, 1}*”. The goal is to 
determine whether there exists an index i G [n— to-|- 1] such that x[z, z-|-to— 1] = y. 
For the purpose of lower bounds, we would consider the simple Boolean function 
defined above. However, some of the algorithms we present can additionally find 
the position of the match f, if it exists. 

We denote the Hamming distance of two strings x, y G {0, 1}” by HD(x, y) 

|{z G [n] : Xi yf yj}|. A relaxed version of pattern matching is the (ji,m,e) 
approximate pattern matching problem, denoted APM„_m_£, in which Bob would 
like to determine whether there exists an index z G [n — to -I- 1] such that x[z, z -|- 
TO — 1] = y, or whether for all z G [rz] , HD(x[z, z -|- to — 1] , y) > em, assuming that 
one of the two holds. 

Notation. Throughout the paper we denote random variables in upper case. For 
a Boolean string x G {0, 1}", jxj denotes the Hamming weight (i.e., the number of 
I’s) of X. log denotes a logarithm to the base 2; In denotes the natural logarithm. 
H 2 {p) = —p^ogp — {1 — p) log(l — p) is the binary entropy function. 

3 Exact Pattern Matching 

In this section we obtain a simple sketching algorithm for exact pattern matching 
and show almost matching lower bounds. Recall that we denote by PM„_m the 
problem in which Alice gets a string x G {0, 1}”, Bob gets a string y G (0, 1}*”, 
and their goal is to find whether there exists an index z G [rz — to -I- 1] such that 
x[z, z -I- TO — 1] = y. 



3.1 Upper Bounds 

First, we show an efficient (randomized) sketching algorithm for the pattern 
matching problem, based on the Karp-Rabin hash function [17]. Next, we show 
a deterministic sketching algorithm for the Boolean version of the pattern match- 
ing problem. 
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Proposition 1. Form < n—logn, there is a one-sided error randomized sketch- 
ing algorithm for the pattern matching problem PMn,m using a sketch of size 
0{n — m). 

Proof. The randomized algorithm is based on the Karp-Rabin method [17]. Let 
t = n— TO+1; we assume in the sequel that t < n/3, as otherwise the proof follows 
trivially by Alice sending x. Let , . . . , x* denote the sequence of t substrings 
of X of length to. Alice and Bob use the shared randomness to pick a (weak) 2- 
universal hash function h : {0, 1}™ ^ [n^j. Alice sends to Bob /i(x^), . . . , h{yf). 
Bob outputs “match found at f”, if h{yd) = h{y). If no such i exists, Bob outputs 
“no match found” . 

This is a one-sided error algorithm: if there is a match, it will surely be output. 
There is a possibility for a false match, though: when x* yf y, but /i(x*) = h{y). 
The probability for a false match is thus at most the probability h has a collision 
between y and any of {x^, . . . ,x*}. A union bound shows that since the range of 
h is large enough, the probability of a collision between y and any is at most 
0(l/n). 

The scheme described above uses a sketch of size Oft log n) = 0{fn—m) log n) . 
A further improvement is possible using the Karp-Rabin hash function: h{b) = 
■ 2'"“*) mod p, where p is a randomly chosen prime in the range [n^]. 
The advantage of this hash function is that the value of /i(x*+^) can be com- 
puted from the value of /i(x*) and from the two bits x^ and x^+m: hfid'^^) = 
((/i(x*) — Xi ■ 2™“^) • 2 -I- Xj_|_m) mod p. Thus, what Alice needs to send is only 
/i(x^), the first t bits of x, and the last t bits of x. Thus, the sketch size goes 
down to 2t -\- O(logn) = 0{n — to). 

Proposition 2. There is a deterministic sketching algorithm for the pattern 
matching problem PM„^m using a sketch of size 2"^. 

Proof. In the deterministic algorithm Alice sends to Bob a characteristic vector 
of length 2™ specifying all the strings of length to that occur as substrings of x. 
Bob outputs “match found” if and only if y is one of the substrings indicated by 
the characteristic vector. 

3.2 Lower Bounds 

We show lower bounds on the sketch size for the pattern matching problem. 
The first one. Theorem 1, deals with the case to > l7(logn). The second one. 
Theorem 2, considers the case to < O(logn). 

Theorem 1. If n < m-\-S2"^, then any 6-error randomized sketching algorithm 
for the pattern matching problem PM„_m requires a sketch of size at least (n — 
TO -I- 1) • (1 — H2{25)), where H 2 f) is the binary entropy function. 

Proof. Using Yao’s Lemma [25], it suffices to exhibit a distribution p, over in- 
stances of and prove that any deterministic sketching algorithm that 

computes PM„,m correctly with probability at least 1 — (5 when running over 
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inputs chosen according to fj, requires a sketch of size at least (n — m + 1) • (1 — 
H2{2S)). 

The distribution fj, is defined as follows. Alice is given a uniformly chosen 
bitstring X G {0, 1}". Bob is given a uniformly chosen substring of X of length 
TO — 1 concatenated with the bit 1 . 

The distributional lower bound w.r.t. /i is proven via a reduction from the 
following version of the indexing function, which we denote by IND„^fc: Alice is 
given a string x G {0, 1}", and Bob is given an index fc + 1 < j < n and a string 
y e {0, 1}^, which is promised to be equal to the substring x[j — k,j — 1]. The 
goal is to compute Xj. 

Let V be the following distribution over instances of IND„^fc. Alice gets a 
uniformly chosen bitstring X in {0, 1}”; Bob gets an index J, which is cho- 
sen independently and uniformly in the interval [k + l,n], and also the bits 

^J-k, ■ ■ ■ 

The following lemma shows the reduction. 

Lemma 1. Any deterministic sketching algorithm U that computes with 

error probability at most S on instances drawn according to p, yields a determin- 
istic sketching algorithm U' that computes IND„^m-i with error probability at 
most 26 on instances drawn according to v and using exactly the same sketch 
size. 

Proof. In the indexing algorithm U' , given an input x G {0, 1}”, Alice sends 
whatever message she would have sent on this input in the algorithm U. Given 
his input (j, y), where m < j < n and y e {0, 1}™“^, Bob simulates the role of 
Bob in the pattern matching algorithm U on the input y o 1, where o denotes 
the concatenation of strings. If the output in 77 is “match found”, Bob outputs 
“I” and otherwise he outputs “0”. 

It is easy to verify that when the input given to 77' is distributed according 
to w, then the input given to 77 in the reduction is distributed according to /i. 
We can thus assume that 77 errs with probability at most 6. 

Fix an input (x, (j, y)) for IND„_m-i, for which the protocol 77 is correct on 
(x,y o 1). If IND„_m-i(x, (j,y)) = 1, then the string yo 1 is a substring of x (at 
position j — m-\- 1), and thus 77' will output “1”, as needed. Suppose then that 
IND„^m-i(x, (j, y)) = 0. Clearly, y o 1 does not equal to the substring of x that 
starts at position j — to -I- 1. Therefore, 77' outputs “0”, unless there is some 
other substring of x that happens to equal to y o 1. We next prove that the latter 
occurs with low probability. 

Define E to be the set of instances (x, (j, y)), for which there exists some 
i yf j, so that the substring x[i, i m — 1] equals y o 1. The proof of Lemma 1 
would follow easily once we prove that Pr(7f) < 6, since 77' errs on an input 
(x, (z,y)) only if either it belongs to 7f or if 77 errs on it. 

Proposition 3. Pr(7f) < <5. 

Proof. Pr(7f) can be rewritten as Pr(3t yf J : X[i, z-I-to — 1] = X[J — to-|- 1, J — 
1] o 1). In order to bound Pr(7f), it would suffice to show that for all choices of 
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m < j < n, Pr(3f ^ j : X[z, z + m — 1] = X[j — to + 1, j — 1 o 1]) < i5. So for 
the rest of the argument, fix such a j. 

Define t j — to + 1. We will show that for all i ^ t, Pr(X[z,z + m — 

1] = X[t, j — 1] o 1) < 1/2™. It would then follow from the union bound that 

Pr(3z yf J : X[z, z + to — 1] = X[j — to + 1, j — 1 o 1]) < (rz — to)/2™ < <5. 

Fix any i ^ t. Suppose, for example, i < t (the case z > t is dealt with 

similarly). X[z, z + to — 1] = X[t, j — 1] o 1 if and only if X[z, z + to — 2] = X[t, j — 1] 
and Xj+m-i = 1. It is easy to verify that the two events are independent and that 
the probability of the second is 1/2. Thus, it suffices to show Pr(X[z, z + to — 2] = 
X[t,j-l]) = l/2™-i. 

We will denote X* = X[z,z + to — 2] and X* = X[t, j — 1]. Let p be the 
length of the longest prefix of X* that does not overlap X* {p can be anywhere 
between 1 and to — 1). Divide X* and X* into s |"(to — l)/p] non-overlapping 
blocks of size p each (except, maybe, for the last one which is shorter). Call 
the corresponding substrings X^ , . . . , X* and X* , . . . , X* , respectively. Note that 
X* = X‘_i for g = 2, . . . , s. X* = X* if and only if X* = X* for all q = 1, . . . , s, 
or equivalently, if and only if all the s blocks of X* equal to X)^ (except, maybe, 
the last one which needs to be a prefix of X)^). The latter event occurs with 
probability 1/2™“^, since the X* and X/ are disjoint substrings of X, and the 
bits of X are chosen independently at random. 

Recall that we assumed the probability II errs on (x, (z, y)) is at most S. Using 
Proposition 3 and applying a union bound, we get that the error probability of 
n' is at most 26, completing the proof of Lemma 1. 

Next, we obtain a lower bound on the one-way communication complexity 
of the modified indexing function. The proof is based on information-theoretic 
arguments. 

Lemma 2. Any one-way deterministic protocol that computes IND„ ^ with er- 
ror probability at most e on inputs chosen according to v requires at least {n — 
/c)(l — H 2 {s)) bits of communication. 

Proof. Fix any such deterministic protocol, and let A(-) denote the function 
Alice applies on her input in this protocol to determine the message she sends 
to Bob. Bob outputs an answer based on the message from Alice and based 
on his input. Thus, using the random variables A(X), J, and Xj_fe, . . . ,Xj_i, 
Bob is able to predict the random variable Xj with probability of error at most 
e. This is exactly the scenario captured by a classical result from information 
theory, called Fano’s inequality (cf. [10]), which implies i? 2 (e) > H(Xj | 

A(X), J, Xj_fe, . . . ,Xj_i). Here H(Z | Y) denotes the conditional Shannon en- 
tropy of the random variable Z given the random variable Y (cf. [10]). By defini- 
tion, the conditional entropy H(Z | Y) equals to X^yH(Z \ Y = y) ■ Pr(P = y), 
where H(Z | U = y) is the entropy of the conditional distribution of Z given the 
event {Y = y}. Expanding over the random variable J, we thus have: 
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H2{s)>-^ ^ H(Xj I A(X),Xj_fe,...,Xj_i,J = j) 

Ti fC 

j=k+l 

1 " 

= T E H(X, M(X),X,_fc,...,X,_i). 

j=k+l 

Conditioning one variable on another can only reduce its entropy. Therefore, we 
can lower bound the j-th term on the righthand side by the conditional entropy 
H(Xj I A(X), Xi, . . . , Xj_i) (we added the random variables Xi, . . . , Xj_fe_i to 
the conditioning). We thus have: 

1 ” 

E H(X, |A(X),Xi,...,X,_i) 

j=k+l 

> ^yH(Xfe+i,...,X„ I A(X),Xi,...,Xfe). 
n — k 

The last transition follows from the chain rule for entropy. Another applica- 
tion of this rule implies that H(Xfe_|_i, . . . ,X„ | A(X),Xi, . . . ,Xfc) = H(Xi, . . . , 
Xn,A(X)) — H(A(X)) — H(Xi,...,Xfe I A(X)). Since A(X) fully depends on 
X = (Xi, . . . ,X„), we have 

H(Xi,...,X„,A(X)) =H(X). 

But, H(X) = n, as X has a uniform distribution on {0, 1}". Since conditioning 
reduces entropy, H(Xi, . . . , X^ | A(X)) < H(Xi, . . . , X^) = k. To conclude: 

H 2 {e) > ^--(n-H(A(X))-fc). 
n — k 

Therefore, H(A(X)) > {n— k){l — H 2 {s)) . Since H(A(X)) is always a lower bound 
on the length of A(X), the lemma follows. 

The proof of Theorem 1 now follows from Lemma 2, Lemma 1, and Yao’s lemma 
[25]. 

For the case when m < O(logn), we have the following lower bound. 

Theorem 2. If n> 2™/^ • m, then any 6-error randomized sketching algorithm 
for the pattern matching problem PM„ „ requires a sketch of size at least 

(l-iL2(<5)). 

Proof. The lower bound follows from a reduction from the indexing problem, 
IND(. The reduction works as follows. Let m be such that t = 2'"/^“^) and 
let n > 2"*/^ • m. Given her input x € {0, 1}*, Alice maps it first into a string 
x' G {0, 1, $ is a special symbol, which we call a “marker” . Let i\, . . . ,ik 

be the positions in which x has a 1. Then x' = zi$Z 2 $ . . . Zfc$$ ... $ where each 
integer ij is written in binary using logt bits and the trailing $’s are used to 
make sure the string is of length n/3. Bob maps his input j G [t] into the string 
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y' = j% in {0, 1, Note that since a marker can match only a marker, xj = 1 

if and only if the pattern y' matches x'. 

In order to complete the reduction, we need to describe a mapping (j) from 
strings over the ternary alphabet into (longer) strings over a binary alphabet, 
so that a pattern y' matches a substring of x' over the ternary alphabet if and 
only if 4>{y') matches a substring of 4>{x'). 4> could be, for example, the following 
mapping: 0 maps to 010, 1 maps to 101, and $ maps to 111. 

4 Approximate Pattern Matching 

In this section we give a nearly-tight sketching lower bound for approximate 
pattern matching. Once again we use the lower bound for the indexing function 
to obtain our lower bound. Recall that we denote by APM„ ^ the problem in 
which Alice gets a string x G {0, 1}”, Bob gets a string y G {0, 1}'", and their goal 
is to determine whether there exists an index i G [n] such that x[i, i + m—1] = y, 
or whether for all i G [n], HD(x[z, i + m — 1], y) > em, assuming that one of the 
two holds. 

Theorem 3. If n < then for any constant 0 < £ < 1/8, any randomized 

sketching algorithm for APM„_m,e requires a sketch of size Q{^). 

Proof. The proof works by a reduction from the indexing function IND^; we 
assume, for simplicity, m divides n, and let t = n/m. We first need to fix a 
collection zi, . . . ,Z2t of 2t binary strings in {0, 1}™ with the following property: 
for any m/2 < s <m, and for any i,j, the prefix of z^ of length s and the suffix 
of Zj of length s have a large Hamming distance, namely, HD(zi [1 , s] , Zj [m — s + 
Ijw]) > A simple probabilistic argument can show that such a collection 
exists as long as t < 2'*''", for some constant 0 < 7 < 1 We will call the last t 
strings in the collection also Oi, . . .0(. 

Suppose n is a sketching algorithm for APM„_m,e- We use it to construct a 
sketching algorithm 77' for INDj. Given her indexing input x G {0,1}*, Alice 
maps it into a string u G {0, 1}”, which is formed by concatenating t strings of 
length m each. The j-th string is zj if xj = 0 and it is Oj if xj = 1. Bob maps 
his input i G [t] into the string v = Oi . 

Alice and Bob now run the algorithm 77 on the inputs u and v. Bob decides 
that Xi = 1 if and only if it is determined that v approximately matches u. 

If Xi = 1, then V = Oi is a substring of u, and therefore the algorithm will 
be correct with high probability. If x^ = 0, then v is not one of the strings 
constituting u. We need to use now the property of the collection zi, . . . ,Z 2* to 
show that no substring of u of length m has small Hamming distance from v. 

Let u^, . . . , u* be the strings (which are taken from zi, . . . , Z2 t) that constitute 
u. Suppose, to the contradiction, u has a substring a of length m such that 

^ Note that the corresponding Hamming distance is the summation of indicators for 
the events (zi)i = (zj)^+m-s taken over £ = l,...,s, and even if i = j, at least 
s/2 > m/2 of them are independent. 
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HD(a,v) < em < m/8. The prefix of a overlaps some u-^ and its suffix overlaps 
Call the overlapping prefix a\ and the overlapping suffix 02 - At least one 
of ai, «2 has to be of size at least m/2. Suppose, for example, it is ai, and let 
s = |ai|. Let vi be the prefix of v of length s. Since the total Hamming distance 
between a and v is at most m/8, also the Hamming distance between a\ and vi 
is at most m/8. But that implies that the Hamming distance between the last s 
bits of u^ and the first s bits of v is at most m/8 even though u^ ^ v. This is a 
contradiction to the property of the collection zi, . . . ,Z 2 t. We conclude that 7T' 
is correct with high probability also when x^ = 0. 

The lower bound now follows from the f2(t) lower bound for indexing. 

Theorem 4. For any constant e > 0, there is a randomized sketching algorithm 
for APM„^m,e with sketch size 0(^e~^ logn). 

Proof. We may assume that the text size is at most n' = (l + e)m, because Alice 
can divide the text x into substrings of length (1 + e)m having an overlap of m 
bits between successive substrings, and then Alice and Bob apply an 0(^ logn) 
sketch independently for of these substrings. If the pattern matches the text, 
then at least one of these substrings must contain that pattern. If the text does 
not contain the pattern, then it suffices to have the algorithm err with probability 
at most 1/n^ on each of the substrings. 

If the text size is n < (I + e)m, Alice simply computes O(logn) random 
inner products Xjrj( mod 2) a la Kushilevitz, Ostrovsky and Rabani [19], and 
sends them to Bob. These inner products are tuned to determine whether the 
Hamming distance is at most em/2 or whether it is at least em, namely, each 
bit rj is chosen independently to be 1 with probability l/(2em) and 0 with 
probability 1 — l/(2em). Since each inner product results in a single bit, the 
sketch size is clearly O(logn). 

It remains to show how Bob uses the results of these 0(log n) inner products 
to determine, with high probability, whether x[i,i + m — 1] = y. Notice that in 
each inner product, 

Fr[rj = 0 for all j < i and for all j > i + m] = (1 — l/(2£m))®'" = 12(1). 

That is, with probability at least some constant, any single inner product with 
X is actually also a random inner product with x[i,i + m — 1], and hence can be 
used by Bob to estimate whether the Hamming distance HD(x[z, z + m — 1], y) is 
at most eml2 or at least em. The details of this estimate are exactly as in [19]; 
in short, in the latter case the probability that the inner product turns out to be 
1 is higher additively by a constant than in the former. By a standard Chernoff 
bound, with high probability (say at least 1 — 1/n^), there are some l7(logn) 
inner products that Bob can use to estimate HD(x[z,z + m — l],y). The proof 
now follows by a union bound over the n' <n possible values of z. 

Remark. The above sketching algorithm is actually stronger than claimed in 
the theorem, as it determines, with high probability, whether there exists an 
index i e [rz] such that HD(x[z, z + m — 1], y) < £mj2, or whether for all z G [n], 
HD(x[z, z + m — 1], y) > em, assuming that one of the two holds. 
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Abstract. In this paper, we study two questions related to the problem of testing 
whether a function is close to a homomorphism. For two finite groups G, H (not 
necessarily Abelian), an arbitrary map f : G ^ H, and a parameter 0 < e < 1, 
say that / is e-close to a homomorphism if there is some homomorphism g such 
that g and / differ on at most e| G| elements of G, and say that / is e-far otherwise. 
For a given / and e, a homomorphism tester should distinguish whether / is a 
homomorphism, or if / is e-far from a homomorphism. When G is Abelian, it 
was known that the test which picks 0(l/e) random pairs x,y and tests that 
f{x) + f{y) = f{x + y) gives a homomorphism tester. Our first result shows 
that such a test works for all groups G. 

Next, we consider functions that are close to their self-convolutions. Let A = 
{ag\g € G} be a distribution on G. The self-convolution of A, A' = {o!g\g G 
G}, is defined by = Yly zeG-yz=x h is known that A = A! exactly 
when A is the uniform distribution over a subgroup of G. We show that there is 
a sense in which this characterization is robust - that is, if A is close in statistical 
distance to A! , then A must be close to uniform over some subgroup of G. 



1 Introduction 

In this paper, we focus on two questions that are related to the problem of testing 
whether a function is close to a homomorphism. 

For two finite groups G, H (not necessarily Abelian), an arbitrary map f : G ^ H, 
and a parameter 0 < e < 1, say that / is e-close to a homomorphism if there is some 
homomorphism g such that g and / differ on at most e|G| elements of G. Define S, the 
probability of group law failure, by 

1 - <5 = Pr [f{x) X f{y) = f{x x y)] . 
x,y 

Define r such that r is the minimum e for which / is e-close to a homomorphism. In 
[4], it was shown that over Abelian groups, there is a constant Sq, such that if 5 < 6q, 
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then the one can upper bound t in terms of a function of <5 that is independent of |G|. 
This yields a homomorphism tester with query complexity that depends (polynomially) 
on 1/e, but is independent of | G| . In particular, the writeup in [4] contains an improved 
argument by Coppersmith [5], which shows that Sq < 2/9 suffices, and that r is upper 
bounded by the smaller root of a;(l — x) = S (yielding a homomorphism tester with 
query complexity linear in 1/e). Furthermore, the bound on Sg was shown to be tight 
for general groups [5]. 

Our first result is to give a relationship between the probability of group law fail- 
ure and the closeness to being a homomorphism that applies to general (non- Abelian) 
groups. We show that for Sg < 2/9, then / is r-close to a homomorphism where 
T = (3 — \/9^^^45)/12 < S/2 is the smaller root of 3x — 6x^ = S. The condi- 
tion on S, and the bound on r as a function of S, are shown to be tight, and the latter 
improves that of [4]. 

Next, consider the following question about distributions that are close to their self- 
convolutions: Let A = {ag\g G G} be a distribution on group G. The convolution of 
distributions A, B is 

C = A*B,Cx= ^ aybz. 

y,z^G\ yz—x 

Let A' be the self-convolution of A, A* A, i.e. a'^ = '^y^z^c-yz=x ttyCtz- It is known 
that A = A' exactly when A is the uniform distribution over a subgroup of G. The 
question considered here is: when is A close to A'l In particular, if dist{A, A') = 
\ ^xeG ~ ^ small enough e, what can be said about Al We show that A 

must be close to the uniform distribution over a subgroup of G, that is, for a distribution 
A over a group G, if dist{A, A* A) < e < 0.0182, then there is a subgroup H of G 
such that dist{A, Uh) < 5e, where Uh is the uniform distribution over H. On the other 
hand, we give an example of a distribution A such that dist{A, A * A) « .1504, but A 
is not close to uniform on any subgroup of the domain. 

A weaker version of this result, with a somewhat more complicated proof, was used 
in the original proof of the homomorphism testing result in [4]. The earlier result was 
never published since the simpler and more efficient proof from [5] was substituted. 
Instead, a separate writeup of weaker versions of both of the results in this paper, by the 
current set of authors, was promised in [4]. This paper is the belated fulfillment of that 
promise, though the earlier results have been strengthened in the intervening time. 

To give a hint of why one might consider the question on convolutions of distribu- 
tions when investigating homomorphism testing, consider the distribution A f achieved 
by picking x uniformly from G and outputting f{x). It is easy to see that the error 
probability <5 in the homomorphism test is at least dist{Af, Af * A f). Unfortunately, 
this last relationship is not in the useful direction. In Remark 2 of Section 3, we 
present a relationship between homomorphism testing and distributions close to their 
self-convolution. 

Related Work: The homomorphism testing results can be improved in some cases: We 
have mentioned that 6g < 2/9 is optimal over general Abelian groups [5]. However, 
using Fourier techniques, Bellare et. al. [1] have shown that for groups of the form 
(Z/2)", (5o < 45/128 suffices. 
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Several works have shown methods of reducing the number of random bits required 
by the homomorphism tests. That is, in the natural implementation of the homomor- 
phism test, 2 log |G| random bits per trial are used to pick x, y. The results of [7, 6, 3, 8 ] 
have shown that fewer random bits are sufficient for implementing the homomorphism 
tests. The recent work of [ 8 ] gives a homomorphism test for general (non-Abelian) 
groups that uses only (1 + o(l)) log 2 |G| random bits. Given a Cayley graph that is 
an expander with normalized second eigenvalue 7 , and for the analogous definitions of 
S, T, they show that for < 1 , r is upper bounded by 4(5/ ( 1 — 7 ) . 

2 Non-Abelian Homomorphism Testing 

In this section, we show that the homomorphism test of [4] works over non-Abelian 
groups as well. As in the Introduction, we define i5, the probability of group law failure, 
by 

1 - 5 = Pr [f{x) X f{y) = f{x X y)] . 
x,y 

We prove the following: 

Theorem If 5 < 2/9 then f is r-close to a homomorphism, where r = [3 — 
\/9 — 24(5]/12 <5/2 is the smaller root ofix — 6 a;^ = 5. 

The rest of this section is devoted to proving the theorem, and showing that the 
parameters of the theorem are tight. 

Pra;,y(*) is the probability of x when x, y are independently selected from G with 
the uniform random distribution. 

Given two finite groups G, H (not necessarily Abelian), and given an arbitrary map 
f : G ^ H (not necessarily a homomorphism) we will construct a map g : G ^ H, 
which (under certain conditions on /) will be a group homomorphism and will agree 
with / on a large fraction of its domain G. 

Given f : G ^ H, with associated 6 < 2/9, we define g : G ^ H hy 

g{a) = majorityxdG [/(« x x) x f{x)~'^] . 

That is, we evaluate the bracketed expression for each x G G, and let g{a) be the value 
most often attained. Define e„, e and r: 

1 - Ca = Pr [/(a X x)/(x)"^ = g(a)] 

X 

e = max Ca 

a 

1 - r = Pr [/(x) = g{x)] 

X 

Lemma 1. If 5 <2/9 then Ca < e where e is the smaller root ofx — x^ = 5. 

Proof: For a G G, define 

[/(® ^ ^ = f{a xy)x f{y)~^] . 

x,y£G 



By rearranging, we have 
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Pa = Pra;.yeG [/(« X y)~^ X f{a X x) = f{y)~^ x f{x)] 

> Pfx.yeG [/(a X j/)-i X /(a x x) = f{y~^ x x) A /(y)"^ x f(x) 

= f{y~^ X x)] . 

? 

Each of the latter two equations is a random instance of the test equation f{u) x f{v) = 
f{u X v), ov equivalently, f(u)~^ x f{ux v) = f{v), so each holds with probability 
1 — 5, and, by the union bound, they both hold simultaneously with probability at least 
1 — 25. So we have 

Pa > 1 - 25 > 5/9. 

If we partition G into blocks 

Ba,z = {a; G G\f{a x x) x f{x)~^ = z} 
with relative sizes ba,z = |Sa,z|/|G|, then 

z 

Pa = y^ bl ^ < max ba,z 
2 

so that maxj;(6a,z) > 5/9, and g{a) = argmaxz{ba,z) is well defined. By definition, 
1 — Ca = maxz(ba,z) >5/9. Since 1 — £„ > 1/2, we also have 

Pa<{l-ear + el 

1 - 25 < 1 - 2ea + 2el 
S>ea- el, 

and since Ca < 1 /2, we conclude that €a < e, the smaller root of a; — a;^ = 5. □ 
Corollary 1. IfS < 2/9 then Ca < 1/3 and e < 1/3. 

Lemma 2. If 6 <2/9 then g is a homomorphism. 

Proof: 

Identity: (/(I) = 1. Immediate since each value x gives /(I x x) x f{x)~^ = 1. 
Inverse: = g{a)~^. There is a one-one correspondence between x satis- 

fying /(a X x) X /(x)-i = g{a) and y satisfying /(a"^ x y) x /(j/)"^ = g(a)"\ 
namely y = a x x. 

Product: g{a) x g{b) = g{a x b). Each of the following three equations holds 
with probability at least 1 — e > 2/3 on random choice of y: 

g{a) = f{a xy)x f{y)~^ 

g{b) = m X /(6-1 X j/)-i 

g{a xb) = /(a xy) X f{b~^ x y)~'^ 
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(In the definition of g, we substitute y = x in the first equation, and y = b x x in the 
second and third.) By the union bound, all three equations hold simultaneously with 
probability at least 1 — 3e > 0; that is, there is at least one value of y satisfying all three 
equations. Substituting one such value of y and combining these three equations, we 
conclude g{a) x g{b) = g{a x b), as desired. □ 

Lemma 3. t < 6 + e. 

Proof: 

T = Pra [/(a) ^ g{a)] 

< P^a,x [f {a) ^ f{a X x) X f{x)-^] + Pra,x [5(a) 7^ f{a x x) x f{x)~^] 

< (5 + Averagea{ea) < <5 + e. 

□ 

Lemma 4. e > 2(r — r^). 

Proof: 

Pr [fix) X fiy)-^ g{x x y-^)] 

x^y 

is the average value of ta (over random choices of a), and so is bounded by e. This 
group law failure will hold at least if either of these two mutually exclusive events 
occurs, since 5 is a homomorphism: 

- f(x) = 9(x)Af(y) g{y)-, 

- fix) ^gix)Afiy) =giy). 

By the independence of x, y, each of the two events has probability r(l — r). So 
e > r(l — r) + (1 — r)r = 2 (r — r^). 



□ 

Corollary 2. If 6 < 2/9 thenr < < 0.2114. 

Proof: If (5 < 2/9 then e < 1 /3, and Lemma 4 implies either r < < 0.2114 or 

T > > 0.7886. The latter is inconsistent with r<(5 + e<5/9 (Lemma 3). □ 

Lemma 5. If 5 <2/9 then S > 3 t — 6 t^. 

Proof: Since g is a homomorphism, the inequality f{x) x f{y) ^ f{x x y) (which 
has probability S) will hold in at least the following three mutually exclusive events: 

- fix) = gix) A fiy) = giy) A fix x y) ^ g{x x j/); 

- fix) = gix) A fiy) f- giy) A f(x x y) = g{x x y)\ 

- fix) gix) A fiy) = giy) A f{x x y) = g{x x y). 



□ 

Lemma 6 . If 5 <2/9 then r is bounded by the smaller root of3x — 6x^ = 6. 
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Proof: Combine Corollary 2 (r < 0.2114) with Lemma 5 (<5 > 3t — 6r^). □ 

This finishes the proof of Theorem 1. 

Example 1. The bound 6 < 2/9 is tight. The following example has <5 = 2/9 and 
e = 1/3, but T = 1 — 1/3^“^ is arbitrarily close to 1. 

/ : 2/3^= ^ Z/3'^-i 

f{U + d) =£,0 < £ < 3'=-\dG {-1,0,1} 

More details are given in the full version [2] . 

Example 2. The bound 3r — 6r^ < <5 is tight. Choose r' with 0 < r' < 1 /3, choose N 
an arbitrarily large odd positive integer, and define / : Z /N Z/2 by 

f{x) = l^r' <^< 2 /. 

More details are given in the full version [2] . 

3 Convolutions of Distributions 

In this section, we show that for a distribution A over a finite group G,if\A — A*A\ < e 
then A is <5-close to the uniform distribution over a subgroup of G. 

We let capital letters A, B, C denote distributions over group G and subscripted 
uncapitalized letters Ux, by denote the probability of a particular element. X, Y, Z, H 
will be subsets of G. 

We let Us denote the uniform distribution on S' C G. 

We let dist{A, B) = ^\A — B\. Note that distances satisfy the triangle inequality, 
i.e., dist{A, C) < dist{A, B) + dist{B, C). Also it is easy to see that dist{A * B,A* 
G) < dist{B, G). 

It will also be convenient to consider a second kind of convolution, 

G = AuB,Cx= ^ aybz. 

y,z^G\ xy—z 

When we have uniform distributions on subsets of equal size, the two convolutions 
enjoy the following relation, the proof of which is given in the full version [2]: 

Lemma 7. Let X,Y,Z be subsets of a finite group G, with lA"! = |K| = jZj = n. 
Then 

dist{Ux, Uy * Uz) = distifly, Uz • Ux)- 
Remark 1. The lemma does not hold for arbitrary distributions, nor for uniform distri- 
butions on subsets of different sizes. 

Overview of Proof : We will embed G in a larger group F = GxZ/N for suitably large 
N, and consider a distribution B induced from A, namely b(^x,j) = a-x/N- This will 
alleviate problems later when we have to round to integers. We show that if B' = B * B 
is close to B, then there is a set AT C such that B is close to Ux- We next show that 
X must be close to a subgroup H of F, and further that this subgroup is of the form 
H = H X Z /N. Then B is close to the uniform distribution on H, and A is close to 
the uniform distribution on H. A bootstrap lemma allows us to claim that once A is 
moderately close to Uh, then it is very close. 
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Expanding the Group: Pick N suitably large. Define F = G x Z/N, with elements 
{{x,j) '■ X G G,j G Z/N} and group law {x,j){y, k) = {xy,j + k). The distribution 
S on is given by A x C/z/n> that is, = ax(l/iV). Defining B' = B * B and 

A' = A* A, it is immediate that dist{B, B') = dist{A, A'). 



B is Close to Uniform on a Subset: Our first theorem shows that if B' = B * B is close 
to B, then there is a set X C F such that B is close to Ux- 

Theorem 2. Let F be a finite group. Let B be a distribution on F for which no element 
has probability more than 1/N. Let 1/S > e > Q be a constant. If dist{B, B * B) < e 
then there is a set X C G such that dist{B, Ux) < e' where e' = 3e + 0{1/N). 
Further, dist{Ux, Ux * Ux) < 6e + O(e^) + 0{1/N). 

Proof: Let B' = B * B. In the rest of the proof, relabel the elements such that 

bi > b 2 > bs > . . ., i.e., 1 corresponds to the element of F with the highest probability 
mass. For given x G G, the N elements b(^x,k), k e Z/N, are equal, so we arrange that 
they are contiguous in this ordering. 

For n > 1, let sum„ = bj be the sum of the n highest probabilities. Also, let 

sum^ = b'j be the sum of the probabilities with respect to B' of the n most likely 

elements with respect to B. 

Let = sum^ + n 6?. It is not hard to see that sum„ > 

U J 

Claim, an > sum^. 

Proof: [of claim] Equivalently, 



1 j>n j xy=Zj 

The right-hand side can be converted to the left-hand side by a carefully selected se- 
quence of “edge swaps”, replacing bkbi + biibk' by bkbk' + bi/bi when k < V and 
k' < 1. Each edge swap does not decrease the sum. Details in the full version [2]. □ 

We will define an X such that dist{B, Ux) is small. Pick r with 1/4 < r < 3/4; 
later we will specify r = 3/5. Select m with sum„_i < r < sum™. Set h = bm- Set 
p = [1 /h\ . Let the distribution U assign weight h to the first p elements, and a weight 
1 — ph < h to the {p + l)st element. Let n = p if bp+i > Up+i, and n = p -I- 1 
otherwise. Let X consist of the first n elements, so that dist{U, Ux) < h = 0(1/A^). 
Also define p = 6„. 

The distribution B differs from U in three places. For i < m, hi > up, define 
P = J2i<7nibi - Ui). For i> n,h> up define 6 = J2t>n(bi ~ Ui). For m < i < n, 
h<up we have /3 + 6 = ~ ^i)- dist{B, U) = /3 + 6. 

For m < i < n we have g < bi < h, so that b/ < {g + h)bt — gh. Similarly for 
i > nwe have 0 < bt < g, so that < gbi. Recall also that mh + (3 = t + 0(1/A^) 
and nh = 1 + 0{1/N). This enables the following computation; 
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e > sum™ - sum'„ > sum„ - am 
= {mh + 13) - {mh + I3Y -m J2l+i ^i~'^ T,i>n 

> {mh + P){1 - mh- (3)- m{g + h) YZi+i bi + m{n - m)gh - mgJ2t>n bi 

= {mh + /3)(1 — mh — (3) — m{g + h){nh — mh — /3 — S) 

+mngh — m?gh — mgS 

= (1 — mh — P){(3) — (1 — mh — (3){mg) + mhS + mg — m'^gh + 0{1/N) 

= (1 - t){( 3) + mh6 + mgf3 + 0{l/N) 

> {1-t)!3 + t5- I35 + 0{IIN) 

= [(1 - T)f3 + t6] + 47(^{[(1 - r)/3 - r<5]2 - [(1 - t)(3 + t 6]^} + 0{l/N) 

> [(1 - r)/3 + t6] - 47(^[(1 - r)/3 + tS]^ + 0{1/N) 

= u-j^ + 0{l/N) 

where t6 = (1 — r)/3 + tS. Now (3 < t + 0{1/N) and 5 < 1 — r so that u = 

(1 — r)/3 + tS < 2r(l — r) is less than the larger root of 



4r(l — r) 



— X + e = 0, 



namely 



[(1 — t)P + t( 5] < 2r(l — r) + 0(l/iV) < 2 t(1 — t) [1 + -\/l — e/r(l — r)] 

(since e/r(l — r) < (l/8)/(l/4x 3/4) < 1), so it must be less than the smaller root: 

[(1 — t)(3 + t( 5] < 2r(l — r)[l — \/l — e/r(l — r)] + 0{l/N), 

remembering the error term. 

Substituting r = 3/5, we have 



2P+3S<— 1-W1-— e +0(l/iV) = 5e + 0(e2) + 0(l/iV). 



By the triangle inequality, 

dist{B, Ux) < dist{B, U) + dist{U , Ux) = (3 + 5 + 0{1/N). 

One can calculate that if e < 1/8 then /3 + <5 < 3e, establishing the theorem. For later 
use, we also note that if e < 0.0182 then (3 + 5 < 2.6e. 

Repeatedly applying the triangle inequality, we can obtain: 

dist{B, Ux * Ux) < dist{B, B * B) + dist{B, B * Ux) + dist{B, Ux * Ux) 

^ e + 2(/3 + (5) . 

To obtain dist{Ux, Ux * Ux) we could apply the triangle inequality again. Instead, 
we recall that B differs from U, and hence Ux, in three pieces, namely (up to errors 
of order 0{l/N)) (3 (before m), —(3 — 6 (between m and n), and S (after n). The 
convolution Ux * Ux is every where bounded by n(l/n)^ = 1/n. So when we change 
B to Ux and monitor the change in dist{-k, Ux * Ux), the first change of (3 is driving 
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us closer to Ux * Ux, while the other two changes of /? + 15 and <5 might drive us further 
away. The net result is 

dist{Ux, Ux * Ux) < dist{B, Ux * Ux) + 5 [— /3 + (/3 + i5) + 5] + 0(l/iV) 

<e + 2p + 36 + 0{l/N) 

<e+f[l-^l-fe\+0{l/N) 

= 6e+0{e^) + 0{l/N). 

This establishes the second part of the theorem. □ 

B Is Close to Uniform on a Subgroup of F: Next we show that if the uniform distri- 
bution on X is close in distance to its convolution with itself, then X is close to some 
subgroup H of F. 

Theorem 3. Let F be a finite group and X a subset of F. Let r = dist(Ux, Ux * 
Ux) = dist{Ux, Ux • Ux)- Ifr <1/9 then there is a subgroup H ofF with |X\/f| + 
\H\X\ < 3r|X|. 

Proof: Let n = | X |. Let V = Ux • Ux so that 

Vx = -^\{{y,z) : y,z e X,xy = z}\. 

If e is the identity element of F, we see Ve = ^, and Vx < ^ for all x G G. 

We need to establish a triangle inequality on quantities such as (ve — Vx)- 

Lemma 8 . Forx,y € F, the quantities {ve — Vx), (ve — Vy), (ve — Vxy) are nonnegative 
and satisfy the triangle inequalities: 

{Ve - Vx) + {Ve - Vy) > {Ve - Vxy) (1) 

{Ve - Vx) + {Ve - Vxy) > {Ve - Vy) (2) 

{Ve - Vy) + {Ve - Vxy) > (Ve ~ Vx) (3) 

Proof: lfvx = kx/ n^, with 0 < kx < n, then there are kx elements z € F such that 

both z and xz are in X ; call such z “good elements” for x. There are n — kx elements z 
such that z € X and xz ^ X; there are n — kx elements z such that z ^ X and xz & X\ 
call the latter two kinds of z “bad elements” for x. The number of bad elements for x is 
2{n — kx) = 2nf{ve — Vx)- If z is neither good nor bad for x it is “neutral” for x. 

For each z G F, consider the three elements z, yz, xyz. If all three are in X, then 
we have found good elements for each of x, y, xy. (Namely, z is a good element for 
y and for xy, and yz is a good element for x.) If exactly two are in X, then we have 
found bad elements for exactly two of x, y, xy, and a good element for the other. (For 
example, if z, xyz G X and yz ^ X, then z is good for xy, z is bad for y, and yz is 
bad for x.) If exactly one of z, yz, xyz is in X, then we have bad elements for exactly 
two of X, y, xy and a neutral one for the other. If z, yz, xyz ^ X then we have found 
neutral elements for all three. The important point is that the “bad elements” come in 
pairs, each time contributing to two of 2(n — kx),2{n— ky),2{n — kxy)- Setting 

p = |{z : z bad for xy and y}\ 
q= \{z : z bad for xy; yz bad for x}| 
r = \{z : z bad for y; yz bad for x}|, 



we find 
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2(n - kx) = q + r 
2(n — ky) = p + r 

2(n- kxy) =p + q. 

This establishes the triangle inequality among 2(n — k^), 2{n — ky), 2{n — kxy), and 
hence the lemma. □ 

Next we show an “excluded middle” result for V. 

Lemma 9. Let t < 1/9. For all x G F, either Vx < 3^ < -^ or Vx > 

Proof: Assume the contrary: for some xG F,^ <Vx < Choose any y G X. 

If xy € X, use triangle inequality (3) to deduce 

(Ve - Vy) + {Ve - Vxy) > {Ve ~ Vx) 

\{Ux)y - Vy\ + \{Ux)xy ~Vxy\> (Ue ~ ) > ^ - ( ^ - ^ ) = ^ ■ 

If xy ^ X, use triangle inequality (1) and {Ux)xy = 0 to deduce 

{Ve - Vx) + {Ve ~ Vy) > {Ve ~ Vxy) 

{Ve Vy) ~\~ Vxy ^ Vx 

\{Ux)y ~ Vy\F \ {Ux)xy ~ Vxy \ ^ Vx > — ■ 

Summing over y G X, 

\-\i^x)y -Vy\ + \{Ux)xy ~ Vxy\] > U ( — ) = 3t. 
yex ^ ^ / 

Then use 

dist{Ux,V) = ^ \{Ux)y - Vy\ 
v&x 

and 

2dist{Ux, V) = ^ \{Ux)z — Vz\ > ^ \{Ux)xy — Vxy\ 
zGF yGX 

to deduce 

(1 + 2)dist{Ux,V) > 3t 
3r = 3dist{Ux, V) > 3r, 

contradicting our hypothesis and establishing the lemma. □ 

The excluded middle gives us a natural candidate for our subgroup H. The follow- 
ing is proved in the full version [2]. 

Lemma 10. Let r < 1/9. Define H = {x G F : Vx > ^}. Then H is a subgroup 
ofF. 

Finally we show that H is close to X. The proof is given in the full version [2]. 

Lemma 11. With r < 1/9 and F[ as above, the symmetric difference between X and 
FI satisfies: 

\H\X\ + \X\H\<j^\X\<3t\X\. 

This establishes the theorem.D 




Non-Abelian Homomorphism Testing 283 



B Is Close to a Subgroup H: We can push these results hack to the distrihutions B: 

Theorem 4. Let F be a finite group. Let B be a distribution on F. Let X C F be a 
subset. Given e, (3, 5 such that: 

- dist{B, B * B) < e; 

- dist{B, Ux) 13 + 5 (as in Theorem 2); 

- r = e + 2/3 + 3h < 1/9, 

then there is a subgroup H of F with dist{B,U pf) < /3 + i5 + 2t/(1 — 3t). The same 
is true if we replace the first condition with 

dist{B, B • B) < e. 

The proof uses Theorems 2 and 3, and is given in the full version [2], 

Reverting to Original Group: The group H respects the block structure of F = G x 
Z/N, in the sense that for all a; ^ e G G and j, k G Z/N, V(x,j) = U{x,k )7 with v 
as defined in Lemma 8, so that (x,j) G H {x,k) G H. Further, one can verify 
that tt(e,fc) > so that if any {x^k) G H with x e, then the entire block 

{ (e, /c) : fc G Z /TV} is in Ff . (Note that if it were the case that there were no (x,k) G H 
with X e, then H = {e}, which is a subgroup of G.) This implies that H is of the 
form 

H = H X Z/N. 

It is obvious that iJ is a subgroup of G, and that 

dist{A, Uh) = dist{B, U pp). 

We tie in with Theorem 2. 

Theorem 5. Let Abe a distribution on the finite group G. Let dist{A, A * A) < e < 
0.0182. Then there is a subgroup H <G G with dist{A, Uh) < 21e. 

Proof: Pass to B and F, with 

dist{B, B * B) < e. 

A computation shows that with e < 0.0182, we have 

2(3+3S<f\l- + 0{1/N) < 5.1e 

P + S< 2.6e. 

From Theorem 2 we have a subset X with 

dist{B, Ux) [3 + 5 < 2.6e. 

Then apply Theorem 4 with r = e + 2/3 + 3(5 < 6.1e < 1/9 to find the subgroup H, 
and use the triangle inequality: 

2r 

dist{B, Upj) < dist{B, Ux) + dist{Ux, Upj) < + 5 + — < 2.6e + 3r < 21e. 

J. oT 
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Reverting to the original distribution, 

dist{A, Uh) = dist{B, Uh) < 21e. 



□ 

Once we have bounds on dist{A, A A) (or dist{A, A • A)) and a subgroup H 
with small dist{A, Uh), we can improve the numerical estimates of dist{A, Uh)- The 
following is proved in the full version of the paper [2]: 

Theorem 6. Given a distribution Aon G and a subgroup HUG with 

dist{A, A* A) = e< 0.06 

dist{A, Uh) = P < 0.4 

then we can conclude 

dist{A, Uh) < 5e. 

Combining the last two results, we have: 

Theorem 7. Let Abe a distribution on the finite group G. Let dist{A, A * A) < e < 
0.0182. Then there is a subgroup HUG with dist{A, Uh) < 5e. 

Proof: From Theorem 5 we have such a subgroup H with dist{A, Uh) < 21e < 

0.3822 < 0.4. Since 0.0182 < 0.06, Theorem 6 applies, giving dist{A, Uh) < 5e. □ 

Example 3. Let G = Z/N with N a large prime integer. Let a„ = r/(240 — |n|) when 
—200 <n<— lorl<n< 200 and a„ = 0 otherwise, where u = 1/55800 is chosen 
to normalize A. Then dist{A, A* A) k- 0.1539, but dist{A, Uh) = 1 — 0(1/-^) for 
any subgroup H of Z/N . 

A gap remains. 

Remark 2. The two notions explored in this paper (homomorphism testing, and dis- 
tributions close to their self-convolution) are related. Given a map (not necessarily a 
homomorphism) f : G ^ H between two hnite groups, we can construct the product 
group 

G X H = {{x,y) : X G G , y G H} 

and a distribution A: 

^(l/\G\ify = f{x) 

^ 0 otherwise 

Then we have the identity 

dist{A, A*A) = Pr[/(x) x f{y) ^ f{x x y)]. 
x,y 

If / is close to a homomorphism g, then A is close to the uniform distribution on the 
subgroup {(a:,(/(x)) : x G G}. Conversely, if / is likely to pass the homomorphism 
test, or equivalently, if A is such that dist{A, A * A) is small enough, then / must be 
close to a homomorphism: The theorem from this section implies that A must be close 




Non-Abelian Homomorphism Testing 285 



to a distribution that is uniform over a subgroup S of G x H. One can show that S is 
such that for each a G G, [S' n {(®7 b) : b G H}\ = 1. Then S can be used to define a 
map g : G ^ H such that 5 is a homomorphism (this can be seen from the underlying 
group structure of S) and such that g agrees with / on most inputs. 

But the correspondence when dist{A, A * 2 I) is larger is not exact: the map / given 
in Example 1 is not at all close to any homomorphism g on all of G, but there is a 
subgroup iJ of GxiT with = 2/3, namely iT = {(3£,£) :0<£< 3^“^}. 

The difference comes because g is required to be a homomorphism on all of G. We 
could relax the requirement, and notice that there is a large subgroup G' of G (namely 
G' = {3^}, with |G'| = |G|/3) and a homomorphism g' : G' ^ H that agrees with / 
on this subgroup. The subgroup H of G x H is associated with G' and g'. 
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Abstract. We continue the investigation of locally testable codes, i.e., 
error-correcting codes for whom membership of a given word in the code 
can be tested probabilistically by examining it in very few locations. 
We give two general results on local testability: First, motivated by the 
recently proposed notion of robust probabilistically checkable proofs, we 
introduce the notion of robust local testability of codes. We relate this 
notion to a product of codes introduced by Tanner, and show a very 
simple composition lemma for this notion. Next, we show that codes 
built by tensor products can be tested robustly and somewhat locally, 
by applying a variant of a test and proof technique introduced by Raz 
and Safra in the context of testing low-degree multivariate polynomials 
(which are a special case of tensor codes). 

Combining these two results gives us a generic construction of codes of 
inverse polynomial rate, that are testable with poly-logarithmically many 
queries. We note these locally testable tensor codes can be obtained from 
any linear error correcting code with good distance. Previous results 
on local testability, albeit much stronger quantitatively, rely heavily on 
algebraic properties of the underlying codes. 



1 Introduction 

Locally testable codes (LTCs) are error-correcting codes that admit highly ef- 
ficient probabilistic tests of membership. Specifically, an LTC has a tester that 
makes a small number of oracle accesses into an oracle representing a given word 
w, accepts if tc is a codeword, and rejects with constant probability if w is far 
from every codeword. LTCs are combinatorial counterparts of probabilistically 
checkable proofs (PCPs), and were defined in [18,24,2], and their study was 
revived in [20]. 

Constructions of locally testable codes typically come in two stages. The 
first stage is algebraic and gives local tests for algebraic codes, usually based on 
multivariate polynomials. This is based on a rich collection of results on “linearity 
testing” or “low-degree testing” [1,3-9,13,14,16-18,20,22,24]. This first stage 
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either yielded codes of poor rate (mapping k information symbols to codewords 
of length exp(fc)) as in [14], or yielded codes over large alphabets as in [24]. 
To reduce the alphabet size, a second stage of “composition” is then applied. 
In particular, this is done in [20,13,11] to get code mapping k information 
bits to codewords of length over the binary alphabet This composition 

follows the lines of PCP composition introduced in [4], but turns out to be fairly 
complicated, and in most cases, even more intricate than PCP composition. The 
one exception is in [20, Section 3], where the composition is simple, but based on 
very specific properties of the codes used. Thus while the resulting constructions 
are surprisingly strong, the proof techniques are somewhat complex. 

In this paper, we search for simple and general results related to local testing. 
A generic (non-algebraic) analysis of low-degree tests appears in [19], and a 
similar approach to PCPs appears in [15]. Specifically, we search for generic 
(non-algebraic) ways of getting codes, possibly over large alphabets, that can be 
tested by relatively local tests, as a substitute for algebraic ways. And we look 
for simpler composition lemmas. We make some progress in both directions. We 
show that the “tensor product” operation, a classical operation that takes two 
codes and produces a new one, when applied to linear codes gives codes that 
are somewhat locally testable (See Theorem 1). To simplify the second stage, we 
strengthen the notion of local testability to a “robust” one. This step is motivated 
by an analogous step taken for PCPs in [11], but is naturally formulated in 
our case using the “Tanner Product” for codes [26]. Roughly speaking, a “big” 
Tanner Product code of block-length n is defined by a “small” code of block- 
length n' = o(n) and a collection of subsets Si,. , Sm C [n], each of size n' . A 
word is in the big code if and only if its projection to every subset Si is a word 
of the small code. Tanner Product codes have a natural local test associated 
with them: to test if a word w is a codeword of the big code, pick a random 
subset Sj and verify that w restricted to Sj is a codeword of the small code. The 
normal soundness condition would expect that if w is far from every codeword, 
then for a constant fraction of such restrictions, w restricted to Sj is not a 
codeword of the small code. Now the notion of robust soundness strengthens 
this condition further by expecting that if w is far from every codeword, then 
many (or most) projections actually lead to words that are far from codewords of 
the small code. In other words, a code is robust if global distance (from the large 
code) translates into (average) local distance (from the small code). A simple, 
yet crucial observation is that robust codes compose naturally. Namely, if the 
small code is itself locally testable by a robust test (with respect to a tiny code, 
of block-length o(n')), then distance from the large code (of block-length n) 
translates to distance from the tiny code, thus reducing query complexity while 
maintaining soundness. By viewing a tensor product as a robust Tanner product 
code, we show that a (logN/loglog A^)-wise tensor product of any linear code 
of length n = poly log iV and relative distance 1 — which yields 

a code of length N and polynomial rate, is testable with poly (log N) queries 
(Theorem 2). Once again, while stronger theorems than the above have been 
known since [6], the generic nature of the result above might shed further light 
on the notion of local testability. 
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Organization. We give formal definitions and mention our main theorems in 
Section 2. In Section 3 we analyze the basic tester for tensor product codes. 
Finally in Section 4 we describe our composition and analyze some tests based 
on our composition lemma. 

2 Definitions and Main Resnlts 

Throughout this paper S will denote a finite alphabet, and in fact a finite field. 
For positive integer n, let [n] denote the set {!,... ,n}. For a sequence x G if” 
and z G [n], we will let Xi denote the zth element of the sequence. The Hamming 
distance between strings x,y G denoted A{x,y), is the number of z G [n] 
such that Xi yf yi. The relative distance between x,y G if", denoted 5{x,y), is 
the ratio A(x.,y')ln. 

A code C of length n over A is a subset of A”. Elements of C are referred 
to as codewords. When A is a field, one may think of A” as a vector space. If 
C is a linear subspace of the vector space A”, then C is called a linear code. 
The crucial parameters of a code, in addition to its length and the alphabet, 
are its dimension (or information length) and its distance, given by A{C) = 
mina;^ygc{Z\(a;, z/)}. A linear code of dimension fc, length n, distance d over the 
alphabet S is denoted an [n,k,d\s code. For a word r G A” and a code C, 
we let 5c{r) = min^jg c{<5(z’, a:)}. We say r is i5'-proximate to C (<5'-far from C, 
respectively) if 5c{r) > 6' (dc(r) > S', respectively). 

Throughout this paper, we will be working with infinite families of codes, 
where their performance will be measured as a function of their length. 

Definition 1 (Tester). A tester T with query complexity q{-) is a probabilistic 
oracle machine that when given oracle access to a string r G A”, makes q{n) 
queries to the oracle for r and returns an accept/reject verdict. We say that T 
tests a code C if whenever r G C, T accepts with probability one; and when 
r ^ C, the tester rejects with probability at least Sc(j’)/2. A code C is said to be 
locally testable with q{n) queries if there is a tester for C with query complexity 
q{n). 

When referring to oracles representing vectors in A”, we emphasize the 
queries by denoting the response of the zth query by r[z], as opposed to rj. 
Through this paper we consider only non-adaptive testers, i.e., testers that use 
their internal randomness R to generate q queries ii, . . . ,iq G [rz] and a predicate 
P : A'^ — > {0, 1} and accept iff P{r[ii ], . . . , r[iq]) = 1. 

Our next definition is based on the notion of Robust POP verifiers introduced 
by [11]. We need some terminology first. 

Note that a tester T has two inputs: an oracle for a received vector r, and a 
random string s. On input the string s the tester generates queries zi, . . . , Zg G [rz] 
and fixes circuit C = Cg and accepts if C(r[zi], . . . ,r[iq\) = 1. For oracle r and 
random string s, define the robustness of the tester T on r, s, denoted p^(r, s), 
to be the minimum, over strings x satisfying C{x) = 1, of relative distance 

of (r[zi], . . . , r[zgj) from x. We refer to the quantity p^(z’) Es[p^(r, s)] as 
the expected robustness of T on r. When T is clear from context, we skip the 
superscript. 
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Definition 2 (Robust Tester). A tester T is said to he c-rohust for a code C 
if for every r G C, the tester accepts w.p. one, and for every r G 27”, Sc{r) < 
c • p'^ir). 

Having a robust tester for a code C implies the existence of a tester for C, 
as illustrated by the following proposition. It’s proof appears in the full version 
of the paper [12]. 

Proposition 1. If a code C has a c-rohust tester T for C making q queries, 
then it is locally testable with 0{c • q) queries. 

The main results of this paper focus on robust local testability of certain 
codes. For the first result, we need to describe the tensor product of codes. 

Tensor Products and Local Tests. Recall that an [n, k, <l\s linear code C may be 
represented by a A: x n matrix M over S (so that C = {xM\x G 27^}). Such a 
matrix M is called a generator of C. Given an [ni, ki,di]s code Ci with gener- 
ator Ml and an [n 2 ,k 2 ,d 2 ]i; code C 2 with generator M 2 , their tensor product 
(cf. [21], [25, Lecture 6, Section 2.4]), denoted C\ ® C 2 Q is the code 

whose codewords may be viewed as rz 2 x ni matrices given explicitly by the set 
{Mff XMi\X G jg well-known that C\ ® C 2 is an [niri 2 , k\k 2 , did 2 ]s 

code. 

Tensor product codes are interesting to us in that they are a generic construc- 
tion of codes with “non-trivially” local redundancy. To elaborate, every linear 
code of dimension k does have redundancies of size 0{k), i.e., there exist subsets 
of t = 0{k) coordinates where the code does not take all possible 27* possible 
values. But such redundancies are not useful for constructing local tests; and 
unfortunately generic codes of length n and dimension k may not have any re- 
dundancies of length o{k). However, tensor product codes are different in that 
the tensor product of an [n, k, d\s code C with itself leads to a code of dimension 
k^ which is much larger than the size of redundancies which are 0(/c)-long, as 
asserted by the following proposition. 

Proposition 2. A matrix r G is a codeword of C\ G C 2 if and only if 

every row is a codeword of C\ and every column is a codeword of C 2 . 

In addition to being non-trivially local, the constraints enumerated above are 
also redundant, in that it suffices to insist that all columns are codewords of C 2 
and only k 2 (prespecified) rows are codewords of Ci. Thus the insistence that 
other rows ought to be codewords of Ci is redundant, and leads to the hope 
that the tests may be somewhat robust. Indeed we may hope that the following 
might be a robust test for Ci G G 2 . 

Product Tester: Pick b G {1,2} at random and i G [ub] at random. 

Verify that r with 6th coordinate restricted to f is a codeword of C^-b- 

While it is possible to show that the above is a reasonable tester for C\®C 2 , 
it remains open if the above is a robust tester for Ci G C 2 . (Note that the query 
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complexity of the test is max{m,n 2 }, which is quite high. However if the test 
were robust, there would be ways of reducing this query complexity in many 
cases, as we will see later.) 

Instead, we consider higher products of codes, and give a tester based on an 
idea from the work of Raz and Safra [23] . Specifically, we let C™ denote the code 
C ® ® C . We consider the following test for this code: 

m 

m-Product Tester: Pick b G \m] and i G [n] independently and uni- 
formly at random. Verify that r with 6th coordinate restricted to i is a 

codeword of C^~^. 

Note that this tester makes queries to test a code of length N = n™. So 

its query complexity gets worse as m increases. However, we are only interested 
in the performance of the test for small m (specifically m = 3,4). We show that 
the test is a robust tester for C™ for every m > 3. Specifically, we show 

Theorem 1. For a positive integer m and [n, k, d\s-code C, such that > 

m-Product Tester is 2^^ -robust for C"”. 

This theorem is proven in Section 3. Note that the robustness is a constant, 
and the theorem only needs the fractional distance of C to be sufficiently large 
as a function of m. In particular a fractional distance of 1 — suffices. Note 
that such a restriction is needed even to get the fractional distance of C™ to be 
constant. 

The tester however makes a lot of queries, and this might seem to make this 
result uninteresting (and indeed one doesn’t have to work so hard to get a non- 
robust tester with such query complexity). However, as we note next, the query 
complexity of robust testers can be reduced significantly under some circum- 
stances. To describe this we need to revisit a construction of codes introduced 
by Tanner [26]. 

Tanner Products and Robust Testing. For integers (n, m, t) an (n, m, t)-ordered 
bipartite graph is given by n left vertices [n], and m right vertices, where each 
right vertex has degree t and the neighborhood of a right vertex j G [m] is 
ordered and given by a sequence = {(.jp, ■ ■ ■ with G [n]. 

A Tanner Product Code (TPC), is specified by an [n,m,t] ordered bipar- 
tite graph G and a code Csmaii C A*. The product code, denoted TPC(G = 

. . . , , Gsmaii) C A , is the Set 

{r e A” I ‘^= , , . . . , ,) e Gsmaii, Vj G [m]}. 

Notice that the Tanner Product naturally suggests a test for a code. “Pick a 
random right vertex j G [m] and verify that r|^^. G Gsmaii-” Associating this test 
with such a pair (G, Gsmaii), we say that the pair is c-robust if the associated 
test is a c-robust tester for TPC(G, Gsmaii)- 

The importance of this representation of tests comes from the composabil- 
ity of robust tests coming from Tanner Product Codes. Suppose (G, Gsmaii) is 
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c-robust and Csmaii is itself a Tanner Product Code, TPC(G', Csmai/) where 
G' is an (d, m', t')-ordered bipartite graph and (G',Gsmai/) is c'-robust. Then 
TPC(G, Gsmaii) has an c • c'-robust tester that makes only t' queries. (This fact 
is completely straightforward and proven in Lemma 3.) 

This composition is especially useful in the context of tensor product codes. 
For instance, the tester for is of the form (G, G^), while G^ has a robust 
tester of the form (G',G^). Putting them together gives a tester for G"^, where 
the tests verify appropriate projections are codewords of G^. The test itself is 
not surprising, however the ease with which the analysis follows is nice. (See 
Lemma 4.) Now the generality of the tensor product tester comes in handy as 
we let G itself be G'^ to see that we are now testing G'® where tests verify some 
projections are codewords of C'^ . Again composition allows us to reduce this to 
a G'^-test. Carrying on this way we see that we can test any code of the form 
G^ by verifying certain projections are codewords of G^. This leads to a simple 
proof of the following theorem about the testability of tensor product codes. 

Theorem 2. Let {Ci\i he any infinite family of codes with Ci a [ni,ki,di]si 
code, with Ui = p{ki) for some polynomial p{-). Further, let ti be a sequence of 
integers such that rrii = 2** satisfies di/ui > 1 — Then the sequence of codes 
|G' = is a sequence of codes of inverse polynomial rate and constant 

relative distance that is locally testable with poly logarithmic number of queries. 

This theorem is proven in Section 4. We remark that it is possible to get 
code families Ci such as above using Reed-Solomon codes, as well as algebraic- 
geometric codes. 

3 Testing Tensor Product Codes 

Recall that in this section we wish to prove Theorem 1. We first reformulate this 
theorem in the language of Tanner products. 

Let G” denote the graph that corresponds to the tests of G™ by the m- 
Product Tester, where G C A”. Namely GJjj has n™ left vertices labelled by 
elements of [n]™. It has m ■ n right vertices labelled {h,i) with b G [to] and 
i G [nj. Vertex (b,i) is adjacent to all vertices {ii, . . . ,im) such that = i- 
The statement of Theorem 1 is equivalent to the statement that (G]jj,G™“^) 
is 2^®-robust, provided > |. The completeness of the theorem follows 

from Proposition 2, which implies G™ = TPC(G]jj, G™”^). For the soundness, 
we first introduce some notation. 

Consider the code Gi 0 • • • 0 Cm, where Ci = [ni,ki,di]s code. Notice 
that codewords of this code lie in The coordinates of strings in 

A"iX 'X"m themselves m-dimensional vectors over the integers (from [ni] x 
••• X [um]). For r G X’”ix-"X"m and ii,...,im with ij G [nfi, let r[ii, . . . ,im] 
denote the (zi, . . . , Zm)-th coordinate of r. For b G [to], and z G [ub], let rf,y G 
y^nix .xri|,_ixn6+ix .xrim the vector obtained by projecting r to coordinates 
whose 6th coordinate is z, i.e., rb^i[ii, . . . , Zm-i] = r[ii, . . . , ib-i,i, it, ■ . ■ , Zm-i]- 

The following simple property about tensor product codes will be needed in 
our proof. It’s proof appears in the full version of the paper [12]. 
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Proposition 3. For b € m} let Ct be an [nbjkb,db]i: code, and let h 

be a set of cardinality at least Ub — db + 1- Let be the code obtained by the 
projection of Cb to lb- Then every codeword d of C'^® ■■■ ® C'^ can be extended 
to a codeword c of C\ ® ■ ■ ■ ® Cm ■ 

Recall that the m-Product tester picks a random b G [m] and i G [n] and 
verifies that rb,i G . The robustness of this tester for oracle r on random 

string s = {b,i) is given by p{r, (b,i)) = 6cm-i(rbp), and its expected robustness 
is given by p{r) = We wish to show for every r that Sc'^{r) < 

2^® • p{r). 

We start by first getting a crude upper bound on the proximity of r to C"* 
and then we use the crude bound to get a tighter relationship. To get the crude 
bound, we first partition the random strings into two classes: those for which 
the robustness p{r, {b, i)) is large, and those for which it is small. More precisely, 
for r G 27” and a threshold r G [0,1], define the r-soundness-error of r to 
be the probability that > Tj when b G [m] and i G [n] are chosen 

uniformly and independently. Note that the y^soundness error of r is at most 
y/p for p = p{r). We start by showing that r is 0(r + e)-close (and thus also 
O(y^) -close) to some codeword of C™. 

Lemma 1. If the r-soundness-error of r is e for r -\- 2e < , then 

Sc-^ir) < 16 • • (r-ke). 

Proof. For every i G [n] and b G [m], fix Cbp to be a closest codeword from 
to rtp. We follow the proof outline of Raz & Safra [23] which when adapted to 
our context goes as follows: (1) Given a vector r and an assignment of codewords 
Cbp G we define an “inconsistency” graph G. (Note that this graph is not 

the same as the graph that defines the test being analysed. In particular G 
is related to the word r being tested.) (2) We show that the existence of a large 
independent set in this graph G implies the proximity of r to a codeword of C"* 
(i.e., Sc^{r) is small). (3) We show that this inconsistency graph is sparse if the 
r-soundness-error is small. (4) We show that the distance of C forces the graph 
to be special in that every edge is incident to at least one vertex whose degree 
is large. 



Definition of G. The vertices of G are indexed by pairs (6, i) with b G [m] 
and i G [n]. Vertex (f>i,ii) is adjacent to (62,12) if at least one of the following 
conditions hold: 

1. Scm-l[rbj^^i^) >T. 

2. 5cm-i{rb^p.f) >T. 

3. 61 yf 62 and Cb^p^ and Cb 2 ,i 2 are inconsistent, i.e., there exists some ele- 
ment j = G [n]'”, with jb^ = i\ and jb 2 = *2 such that 

where G [n]"*”^ is the vector j with its 6cth 

coordinate deleted. 
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Independent Sets of G and Proximity of r. It is clear that G has mn vertices. 
We claim next that if G has an independent set / of size at least m{n — d) + d+l 
then r has distance at most 1 — (|/|/(mn))(l — r) to C™. 

Consider an independent set J = Ji U • • • U in G with Ih of size Ub being the 
set of vertices of the form {b,i),i G [n]. W.l.o.g. assume ni > • • • > n^- Then, 
we have ni, ri2 > n — d (or else even if ni = n and U2 = n — d we’d only have 
< n + (m — l)(n — d)). We consider the partial vector r' G ^Lxnx -xn 
defined as r'[ij2, ■ ■ ■ Jm] = Ci,i[j2, ■ • -Jm] for i G h, and j2, • ■ • , jm G [n]. We 
show that r' can be extended into a codeword of G™ and that the extended word 
is close to r and this will give the claim. 

First, we show that any extension of r' is close to r: This is straightforward 
since on each coordinate i G I\, we have r agrees with r' on 1 — t fraction of 
the points. Furthermore li/n is at least \I\/{mn) (since ni is the largest). So we 
have that r' is at most 1 — (|/|/(mn))(l — t) far from r. 

Now we prove that r' can be extended into a codeword of G"*. Let Cb = C\i^ 
be the projection (puncturing) of G to the coordinates in lb- Let r" be the 
projection of r' to the coordinates in G x J2 x [n] x • • • x [n]. We will argue below 
that r" is a codeword of Gi 0 G2 C) G™“^, by considering its projection to axis- 
parallel lines and claiming all such projections yield codewords of the appropriate 
code. Note first that the restriction of r' to any line parallel to the 6-th axis is 
a codeword of G, for every 6 G { 2 , . . . , to}, since rj ^ is a codeword of G™“^ for 
every i G I\. Thus this continues to hold for r" (except that now the projection to 
a line parallel to the 2 nd coordinate axis is a codeword of G2). Finally, consider 
a line parallel to the first axis, given by restricting the other coordinates to 
{i2, ■ ■ ■ ,im), with i2 G l2- We claim that for every ii G I\, r"[ii, . . . ,im] = 
C2,i2 [G, ■ • ■ , im]- This follows from the fact that the vertices ( 1 , if) and ( 2 , 12) are 
not adjacent to each other and thus implying that and 02,^2 are consistent 
with each other. We conclude that the restriction of r" to every axis parallel 
line is a codeword of the appropriate code, and thus (by Proposition 2 ), r" is a 
codeword of Ci®C2®G™~'^ . Now applying Proposition 3 to the code G\®G™~^ 
and its projection Gi 0 G2 <8> G™”^ we get that there exists a unique extension 
of r" into a codeword d of the former. We claim this extension is exactly r' 
since for every i G I\, c\ = r'[i,j,k]. Finally applying Proposition 3 one 

more time, this time to the code G™ and its projection G\ , we find that 

r' = d can be extended into a codeword of the former. This concludes the proof 
of this claim. 



Density ofG. We now see that the small r-soundness-error of the test translates 
into a small density 7 of edges in G. Below, we refer to pairs (6, i) with 6 G [to] 
and i G [n] as “planes” (since they refer to (to — l)-dimensional planes in [n]™) 
and refer to elements of [n]'" as “points”. We say a point p= (pi, . . . ,Pm) lies on 
a plane (6, i) if pb = i. Now consider the following test: Pick two random planes 
(61, ii) and (62,12) subject to the constraint 61 yf 62 and pick a random point p 
in the intersection of the two planes and verify that is consistent with r[p\. 
Let K denote the rejection probability of this test. We bound n from both sides. 
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On the one hand we have that the rejection probability is at least the prob- 
ability that we pick two planes that are r-robust and incident to each other 
in G (which is at least — 2e) and the probability that we pick a point on 
the intersection at which the two plane codewords disagree (at least (d/n)™”^), 
times the probability that the codeword that disagrees with the point function 
is the first one (which is at least 1/2). Thus we get k > ~ ■ 

On the other hand we have that in order to reject it must be the case 
that either d(^m-i(r{,j_ij) > r (which happens with probability at most e) or 

< T and p is such that and disagree at p (which hap- 

pens with probability at most t). Thus we have k < t + e. Putting the two 
together we have 7 < ^2e -I- {t + e)^ • 

Structure of G. Next we note that every edge of G is incident to at least one 

high-degree vertex. Consider a pair of planes that are adjacent to each other in 
G. If either of the vertices is not r-robust, then it is adjacent to every vertex of 
G. So assume both are r-robust. 

W.l.o.g., let these be the vertices (1, i) and (2, j). Thus the codewords ci,j and 
C2j- disagree on the (m — 2)-dimensional surface with the first two coordinates 
restricted to i and j respectively. Now let S' = { {k^, . . . , km) \ Ci^i [j, /ca, . . . , km] 
C 2 j[i, ks,. ■ ■ , km]} be the set of disagreeing tuples on this line. By the distance 
of we know |S| > But now if we consider the vertex (6, fcb) in G 

for b € {3,..., to} and kb such that there exists k\, . ■ ■ ,km -2 satisfying k = 
{ki , . ■ ■ , km- 2 ) G S, it must be adjacent at least one of (l,f) or (2,j) (it can’t 
agree with both at the point (t,j, fc). Furthermore, there exists d such kfs for 
every 6 G {3, . . . , to}. Thus the sum of the degrees of (1, i) and (2, j) is at least 
(to — 2)d, and so at least one has degree at least (to — 2)dj2. 

Putting It Together. From the last paragraph above, we have that the set of 
vertices of degree less than (to — 2)dj2 form an independent set in the graph G. 
The fraction of vertices of degree at least (to — 2)d/2 is at most 2{jmn) / {{m — 
2)d). Thus we get that if mn- (1 — 2(777171 ) /((to — 2)d)) > m(n — d) + d+l, then 
r is (5-proximate to C"" for S < r-|-(l — r)- 2 ( 777777 ) /((to — 2)d). The lemma 
now follows by simplifying the expressions above, using the upper bound on 7 
derived earlier. The calculations needed to derive the simple expressions can be 
found in the full version of this paper [12]. 

Next we improve the bound achieved on the proximity of r by looking at 
the structure of the graph Gm (the graph underlying the 777-Product tester) 
and its “expansion” . Such improvements are a part of the standard toolkit in 
the analysis of low-degree tests based on axis parallel lines (see e.g., [7, 6, 16, 
17] etc.) We omit the proof of this part from this version - it may be found in 
the full version of this paper [12]. We state the resulting lemma and show how 
Theorem 1 follows from it. 
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Lemma 2. Let m be a positive integer and C he an [n,k,d\s code with the 
property d™-~^/n™-~^ > If r & L’”™ and c G C"" satisfy S(r,c) < j then 
c) < 8p(r). 



( \ 2m 

) . We will prove that the m-Product 

Tester is c-robust for C"”. Note that c < 2^® as required for the theorem, and 
yj < min{^ • (^)’” , ^ • (^)’””'} (as will be required below). 

The completeness (that codewords of C™ have expected robustness zero) 
follows from Proposition 2. For the soundness, consider any vector r G and 
let p = p{r). If p > 1/c, then there there is nothing to prove since 6c'^{r) < 1 < 
c- p. So assume p < 1/c. 

Note that r has yf^soundness-error at most ,Jp. Furthermore, by the as- 



sumption on p, we have 3y/p < and so, by Lemma 1, we 

have Scrr^ir) < 16- (f • 2 • y/p. Now using y/p < • (^)™ \ we get 

Sc^{r) < j. Let u be a codeword of C™ closest to r. We now have 6{r, u) < | 

and ^ and so, by Lemma 2, we get 6c^{r) = 6{r,v) < 8p. This 

concludes the proof. 



4 Tanner Product Codes and Composition 

In this section we define the composition of two Tanner Product Codes, and 
show how they preserve robustness. We then use this composition to show how 
to test C"” using projections to All proofs of Lemmas in this Section appear 
in the full version of the paper [12]. 

Recall that a Tanner Product Code is given by a pair (G, Gsmaii)- We start by 
defining a composition of graphs that corresponds to the composition of codes. 

Given an {N, M, D)-ordered graph G = {£i, . . . ,£m} and an additional 
{D, TO, d)-ordered graph G' = their Tanner Composition, denoted 

G©G', is an (iV, M -to, d)-ordered graph with adjacency lists G [M],j' G 

[to]}, where .. 

Lemma 3 (Composition). Let Gi be an {N, M, D)- ordered graph, and Gi C 
he a linear code with C = TPC(Gi,Gi). Further, let G 2 be an {D,m,d)- 
ordered graph and C 2 C he a linear code such that Gi = TPC(G 2 ,G 2 ). 
Then C = TPC(Gi©G 2 , G 2 ) (giving a d-query local test for C). Furthermore if 
(Gi,Gi) is Cl -robust and {G 2 ,C 2 ) is C 2 ~robust, t/ien (Gi©G 2 , G 2 ) is ci-C 2 ~robust. 

We continue by recasting the results of Section 3 in terms of robustness of 
associated Tanner Products. Recall that G])j denotes the graph that corresponds 
to the tests of G'" by the m-Product Tester, where G C A”. 

Note that G])j can be composed with and so on. For to' < to, define 

= G;/ if to' = TO - 1 and define G” = G” ©G” otherwise. Thus 

we have that C™ = TPC(G[/ G"' ). The following lemma (which follows 
easily from Theorem 1 and Lemma 3 gives the robustness of (G" 2 ,G^). 
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Lemma 4. Let C be an [n,k,d]s code with {d— 1/n)^ < Then (G 4 2 ,C'^) is 
2^“^ -robust. 

Finally we define graphs i?” so that = TPC(i/”, C'^). This is easily done 

recursively by letting H 2 = GJ 2 and letting i/” = G” 2 ©^t"-i for t > 2. We 

now analyze the robustness of (i^", G^). 

Lemma 5. There exists a constant c such that the following holds: Let t be an 
integer and C be an [n, k,d\s code such that d— 1 > (1 — j^)' n. Then {Hf, G^) 
is c* -robust. 

We are ready to prove Theorem 2. 

Proof (Theorem 2). Let c be the constant given by Lemma 5. Fix i and let 
G = Ci, n = Hi etc. (i.e., we suppress the subscript i below). Then G™ is an 
[N,K,D]q code, for N = n"*, K = k'^ and D = d™. Since d/n > 1 — we 
have G™ has relative distance d™" /n^ > Furthermore, the rate of the code is 
inverse polynomial, i.e., N = n'" = (p(fc))™ < poly(fc'") = poly(iF). Finally, we 
have G"* = TPC(iF[”g^ G^), where (dLi”g^^,G^) is a c'°®2 ™_robust tester for 
G™ and this tester has query complexity O(n^). From Proposition 1 we get that 
there is a tester for G that makes ™)) = poly log queries. 
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Abstract. Motivated by an open problem recently suggested by Gol- 
dreich et al., we study truthful implementations of a random binary 
function supporting compound XOR queries over sub-cubes of the hy- 
percube {0,1}". We introduce a relaxed model of an implementation, 
which we call a stateful implementation, and show how to implement 
the desired specification in this model. The main technical construction 
is an algorithm for detecting linear dependencies between n dimensional 
hypercubes, viewed as characteristic vectors in . Using coding the- 

oretic techniques, we first exhibit a randomized algorithm for detecting 
such dependencies. We then show how a recent approach by Raz and 
Shpilka for polynomial identity testing in non-commutative models of 
computation can be applied to obtain a deterministic algorithm. 



1 Introduction 

In a recent paper, Goldreich, Goldwasser and Nussboim [3] initiated the study of 
efficient pseudorandom implementations of huge random objects - objects so big 
that they cannot be represented using bounded resources, such as randomness or 
time (in particular, these objects have size that is exponential in the running time 
of the applications), but for which we can obtain approximations good enough 
for many algorithmic and cryptographic applications. A celebrated example from 
cryptography is the construction of a pseudo-random function from any one way 
function [2]: Even though a truly random boolean function on n input bits cannot 
be specified by fewer than 2" random bits, in many cryptographic applications 
this infeasible object can be approximated by a pseudo-random function that can 
be specified using only poly(n) random bits and evaluated on arbitrary inputs 
in poly(n) time. 

1.1 Stateful and Stateless Implementations 

Since the work of Goldreich et al. is somewhat motivated by cryptography, they 
consider only “stateless” implementations of huge random objects. An imple- 
mentation refers to a polynomial-time (oracle) machine that computes the huge 
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object; in the case of a random boolean function, the machine takes as input a 
string of n bits and outputs the value of the function at that point. In a stateless 
implementation I, the answer to a query posed to I cannot depend on queries 
previously seen by I and their answers. This property of the implementation is 
often important in cryptographic settings, where multiple parties share the same 
copy of the huge object in question. For example, a common technique in cryp- 
tography is to design protocols in the random oracle model, where each party 
has access to the same infinite sequence of random bits. To obtain an implemen- 
tation of the protocol, one replaces the random oracle with a pseudo-random 
function. As we want the parties in the protocol to share the same pseudo- 
random function, it is important that the implementation of the random oracle 
by a pseudo-random function be independent of the queries seen by a particular 
party in the protocol; namely, the implementation must be stateless. 

In addition to cryptography, Goldreich et al. also consider algorithmic ap- 
plications of huge random objects. For example, they imagine a scenario where 
one wants to run experiments on, say, random codes. They observe that global 
properties of these codes, such as having large minimum distance, may not be 
preserved when the randomness of the code is replaced by a pseudo-random gen- 
erator. This leads to the problem of implementing huge random objects that are 
guaranteed to preserve a certain property, such as codes with good minimum 
distance. 

Unlike in the cryptographic setting, it is not clear that a stateless imple- 
mentation (the only type allowed by the model of Goldrech et al.) gives any 
advantage over a “stateful” one. In other words, it may be possible to do more 
by generating the desired huge random object “on the fly” rather than subscrib- 
ing to an implementation predetermined by the random tape of our machine. In 
particular, it would be interesting to know whether there exists a natural specifi- 
cation that allows a stateful implementation but not a stateless one. We suspect 
that the specification considered in this paper, suggested for study by Goldreich 
et al. - a random boolean function supporting XOR queries over hypercubes - 
may provide a separation between stateful and stateless perfect implementations 
in the random oracle model. 

1.2 Random Functions Supporting Complex Queries 

Goldreich et al. observe that, assuming the existence of one-way functions, if 
a specification can be close-implemented^ in the random oracle model, then it 
can also be implemented by an ordinary probabilistic polynomial-time machine. 
Moreover, this transformation preserves truthfulness^: Namely, to obtain a truth- 

^ In fact, it is sufficient that the specification be pseudo-implementable. Note that the 
terms close-implementable, pseudo-implementable and truthful are technical terms 
defined in [3]. 

^ Intuitively, truthfulness requires that an implementation of Type T objects gen- 
erates only objects of Type T. In particular, a random function is not a truthful 
implementation of a random permutation even though they are indistinguishable to 
a computationally bounded adversary. 
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ful pseudo-implementation of a huge random object, it is sufficient to construct 
such an implementation in the random oracle model. This is a common technique 
in cryptography, used among other things in the construction of pseudo-random 
permutations [4]. Though the transformation is only shown to hold for stateless 
implementations, we observe that it also works for stateful ones (this is because 
the definition of pseudo-randomness in the context of pseudo-random functions 
allows from stateful adversaries). 

In particular, this observation implies that random functions have a trivial 
truthful pseudo-implementation. However, one may ask whether it is possible 
to truthfully implement random functions supporting queries beyond evalua- 
tion on arbitrary inputs. One variant proposed by Goldreich et al. asks for the 
implementation of a random function / : {0,1}” ^ {0,1}, augmented with 
queries regarding the XOR of the values of / on arbitrary intervals of {0, 1}” 
(with respect to the lexicographic ordering of n bit strings.) Note that a trivial 
implementation of / in the random oracle model cannot hope to answer such 
queries efficiently, as they may involve XORing exponentially many bits. How- 
ever, Goldreich et al. show how, using a suitable data structure, one can obtain 
a stateless perfect implementation of / that answers queries in O(n^) time. As 
perfect implementations are always truthful, this construction yields a truthful 
pseudo-implementation by an ordinary machine. 

We observe that a simpler construction for implementing random functions 
supporting interval-XOR queries with 0(n) running time can be achieved as 
follows: let /' : {0, 1}” ^ {0, 1} be the random oracle, and return /'(a— 1)0/'(/3) 
as the answer to the query (a, /?), corresponding to the value ®a<x<j 3 /(^) ■ Here, 
a— 1 denotes the n-bit binary string that immediately precedes a in lexicographic 
order, and we specify /'(O" — 1) = 0. The underlying idea is a simple change of 
basis: instead of specifying a random function / by its values at all x G {0, 1}", we 
specify the value of / using the values fiu)- Note that this implementation 

makes only 2 queries into the random oracle per interval-XOR query, which is 
optimal (in an amortized sense). Unfortunately, this construction, unlike that 
by Goldreich et al., does not yield a truthful close-implementation of random 
functions supporting any symmetric interval query. 

As a follow-up to their work on interval queries, Goldreich et al. propose the 
following more general question: 

Open Problem. [3] Provide a truthful close-implementation in the random 
oracle model of the following specification. The specification machine defines 
a random function / : {0,1}” ^ {0,1}, and answers queries that succinctly 
describe a set S, taken from a specific class of sets, with the value 
A natural case is the class of sub-cubes of {0, 1}”; that is, a set S is specified by 
a pattern cr in {0, 1, *}” such that S is the set of points in {0, 1}” that match 
the pattern a. 

We suspect that the technique of Goldreich et al. for implementing a random 
function supporting interval queries does not extend to the case of hypercube 
queries, though we have not been able to show a negative result confirming our 
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intuition. Instead, we show how to obtain a stateful perfect-implementation of 
this specification in the random oracle model. 

1.3 Main Contributions 

Our main contributions are the following: 

1. We propose a notion of stateful implementations of huge random objects and 
reduce the problem of constructing a stateful implementation of a random 
binary function supporting compound XOR queries over sub-cubes of the 
hypercube {0, 1}” to an algorithmic problem of hypercube linearity testing. 

2. We then present two algorithms for hypercube linearity testing: a random- 
ized algorithm using coding theoretic techniques, and a deterministic based 
on non-commutative polynomial identity testing. It follows from the first al- 
gorithm that there is a stateful close implementation of the afore-mentioned 
specification in the random oracle model, and from the second, a stateful 
perfect implementation. 

In fact, the second algorithm subsumes the first, but we still include the latter 
as we feel that the technique used is fairly insightful and the analysis can in fact 
be used to construct quantum states of provably superpolynomial tree size. 

2 Preliminaries 

Let a G {0, 1, *}”. The hypercuhe^ H{a) is the set of all x G {0, 1}” that match 
the string a, namely such that 

{ 0, if a[i] = 0 

1, if a[i] = 1 

0 or 1, if a[i] = *. 

As in Goldreich et ah, we specify the huge random object in question by a 
computationally unbounded probabilistic Turing machine that halts with prob- 
ability one. The huge random object is determined by the input-output relation 
of this machine when the random tape is selected uniformly at random from 
{ 0 , 1 }“. 

A Random Function Supporting XOR Queries on Hypercubes 

INPUT A query a G (0, 1, *}” 

RANDOM TAPE A Sequence of functions /i, / 2 , ■ • ■, where fk ■ (0, 1}^ ^ (0, 1} 
OUTPUT The value f\a\(x) 

We are interested in efficient implementations of this specification, namely 
ones that can be obtained by Turing machines that run in time polynomial in 
the length of the input. 

In [3] and in the introduction, we use the term sub-cubes of the hypercube {0, 1}". 



3 
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2.1 Stateful Implementations 

Following Goldreich et al., we say machine / is a (stateful) implementation spec- 
ification S with respect to a machine M of that makes queries to a protocol 
party, if (1) On the qth query x, I runs in time polynomial in q and x, and (2) 
The distribution Ds of the transcript M(l”) S' is indistinguishable from the 
distribution Dj of M(l") I. Specifically: 

1. If Ds and Dj are identical for every M, we say that / perfectly implements 

S; 

2. If Ds and Dj have negligible statistical difference for all M that 

make poly(n) queries, we say that / closely implements S; 

3. If Ds and Di are indistinguishable for all M that run in polynomial time, 
we say that I pseudo-implements S. 

An implementation / is truthful with respect to S if for every sequence of 
queries xi, . ■ . , Xq, the support of the distribution I{xi), . . . ,I{xq) is contained 
in the support of S(a;i), . . . , S{xq); namely, if the implementation never provides 
answers that are inconsistent with the specification. Note that perfect implemen- 
tations are always truthful. An oracle Turing machine D is an implementation 
of S in the random oracle model if the distribution of M(l") ^ S' is indistin- 
guishable from the distribution of M(l") ^ over a random oracle R. As for 
stateless implementations (see Theorem 2.9 of [3]), we have the following: 

Proposition 1. Suppose that one-way functions exist. Then any specification 
that has a pseudo-implementation in the random oracle model also has a pseudo- 
implementation by an ordinary machine. Moreover, if the former implementation 
is truthful then so is the latter. 

Since perfect implementations are always truthful, for our purposes it will be 
sufficient to provide a perfect implementation of a random function supporting 
queries on hypercubes in the random oracle model. The heart of this implemen- 
tation consists of an algorithm for the problem of hypercube linearity testing, or 
Hypercube-Lin, which we describe next. 

2.2 Reduction to Hypercube Linearity Testing 

We call collection of hypercubes Hi, . . . , Hm C {0, 1}" linearly dependent if there 
exists a non-empty set S C {1,2,..., m} such that for all points x G (0, 1}”, x 
is contained in an even number of hypercubes amongst the subset of hypercubes 
{Hi I i G S}. For any such set S, we write ~ Equivalently, we 

may view each hypercube Hi as a, vector in , where F 2 = (0, 1} is the 

two element field and Hi[x] = 1 if a; G i?i, and 0 otherwise. In this notation, 
linear independence between hypercubes translates into linear independence of 
the corresponding vectors. 

Hypercube-Lin: Given q strings a\,...,aq G {0,1,*}”, accept iff the hyper- 
cubes H{ai), ... ,H{aq) are linearly dependent. 
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Note that standard techniques for testing linear independence, such as Gaus- 
sian elimination, do not apply directly for this problem because we are dealing 
with vectors whose length is exponential in the size of the input. However, we 
will still be able to show the following: 

Theorem 1. There is a deterministic polynomial-time algorithm for the prob- 
lem Hypercube-Lin. 

In fact, we will begin with a randomized algorithm for Hypercube-Lin, 
which nicely illustrates the coding theoretic nature of the hypercube linearity 
testing problem. We then argue that the test performed by the algorithm can 
be viewed, in some sense, as an application of polynomial identity testing. Even 
though we don’t know, in general, how to derandomize polynomial identity test- 
ing, in our case the derandomization can be performed using a recent identity 
testing algorithm for non-commutative formulas of Raz and Shpilka [5] . 

Theorem 2. There exists a stateful perfect implementation of a random func- 
tion supporting XOR queries on hypercubes in the random oracle model. 

Proof. Let A be the algorithm from Theorem 1, and R be the random oracle. 
First, we consider the following (promise) search problem: 

Input Strings Oi, . . . , a, S {0, 1, *}" and b G {0, 1, *}", such that the hypercubes 
H{a \), . . . , H{aq) are linearly independent 

Problem If H{b) is linearly independent from H{ai), . . . , H{aq), output 0 . 
Otherwise, output coefficients ci, . . . , Cg G F 2 such that H{b) = CiH{ai). 

It is not difficult to see that, by a self reduction argument, we can obtain a 
polynomial time algorithm A' for this problem using black box access to A (in 
fact, 2q invocations of A suffices). With this in hand, we implement a random 
function supporting XOR queries on hypercubes as follows: After seeing queries 
ai, . . . , flg-i, the implementation keeps track of a subset of queries {ofe : k G B}, 
where B C [g — 1] is chosen such that the subset of hypercubes {H{ak) : k G B} 
form a basis for the set of vectors {H{ak) G : fc G [q — 1]}. On query Og, 

we run the algorithm A' on inputs {ofe : fc G R} and Og. If the algorithm returns 
0 , then we return R{\B\ -\- 1), which is a fresh random bit, and add Oq to the 
set B. Otherwise, the algorithm N outputs coefficients ci,...,Cg_i G F 2 , and 
we return the value X)l=i CiR{i)- 

We show this is a perfect implementation, by induction on q. Let us assume 
that the specification transcript and implementation transcript are statistically 
indistinguishable after q — 1 queries. At query g, there are two possibilities: If 
H{aq) is linearly dependent in H{ai ), . . . , H{aq-\), then both in the implemen- 
tation and in the specification the answer to the gth query is determined by the 
previous answers, so by the inductive hypothesis the new transcripts are indis- 
tinguishable. If H{aq) is linearly independent in H{ai), . . . , H{aq-i), then the 
answer to the gth query in the specification is statistically independent from all 
previous answers. By construction, this is also true in the implementation, so 
again, the new transcripts are indistinguishable. 
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3 Algorithms for Hypercube Linearity Testing 

In this section, we present two algorithms for Hypercube-Lin, thereby com- 
pleting our implementation of a random function supporting XOR queries on 
hypercubes. 

3.1 A Randomized Algorithm for Hypercube-Lin 

Let F = F 20 be a field of characteristic 2 of size 2^, where s = [log 3n] . For each 
a € {0, 1, *}”, we define a polynomial Pa = z\Z 2 • • ■ Zn over F[xi, ?/i, . . . , Xn, Vn], 
where: 

{ Xi if Oi = 0 

Ui if a* = 1 

Xi + yi if a* = * 

Note that H{a) is a homogeneous polynomial of total degree exactly n, and 

that Pa = J2x&H(a)P^- 

Lemma 1. Let F be any field of characteristic two. Then H{ai) + - ■ ■ + H(am) = 
0 if and only if Pai + • • • + Pam ts the zero polynomial inV[xi,yi, . . . ,a;n,yn]- 

Proof. Each point x in {0, 1}” is represented by a unique monomial Px from 
F[xi, 2 / 1 , . . . , Xn, yn]- Therefore, each point x in {0, 1}" appears an even number 
of times in iL(oi), . . . , H{am) iff each of the corresponding monomials Px has an 
even integer coefficient in H — • +Pam- addition, since F has characteristic 

2, the latter condition is equivalent to + • • • + Pam being identically zero. 

We may now define a binary encoding Cu{a) of hypercubes H{a), which is 
obtained by concatenating the Reed-Muller code associated with the polynomial 
Pa with the Hadamard code. More precisely, given any a G we define 

Cff(a) : {0, ^ {0, 1} as follows: 

CH(a){xi,yi, . ■ .,Xn,yn,T) = {pa{xi,yi , . . . , Xfm ym)-j ) 

using the first 2ns bits of the input to Cu{a) to pick x\,yi, . . . ,Xn,yn G F, and 
the remaining s bits to pick r G {0, 1}®. 

Lemma 2. {Cfj(^a) I a G {Oj !> *}”} is a binary encoding of hypercubes in {0, 1}” 
with the following properties: 

1. (large distance) It has relative distance 1/3. 

2. (locally encodable) There is a O(nslogs) algorithm that computes CH(a)(j’) 
on input a and r G {0, l}(2"+i)s, 

3. (linearity) For all a, o' G {0, 1, *}”, CH{a)+H(a') = CH(a) + CH(a')- 

Proof. By the Schwartz-Zippel Lemma [6,7], for any o yf o' G {0,1,*}", Pa 
and Pa' evaluate to different values on at least a 2/3 fraction of values in F^". 
Upon concatenating with the Hadamard code, the minimum relative distance 
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becomes 1/3. Next, observe that we can write down pa and evaluate pa at any 
input in time O(nslogs). It follows that we also compute CH(a){T) in time 
O (ns logs). Finally, linearity follows from the fact that both the encoding as 
polynomials and the Hadamard code are linear. 



Randomized Algorithm for Hypercube-Lin 

1. Fix £ = 0{q). Choose ri, . . . , G {0, uniformly at random. 

2. Construct the q x £ matrix M over F 2 , where My = CH{ai){'i'j), that is: 



M = 



/ CH{ai){ri) CH(ai){r2) ■ ■ ■ CH(ai){ri) 

CHia2)i'ri) CH{a2){’r2) ■ ■ ■ CH{a2){ri) 



V CH{a^){rx) CH(a,){r2) ■ ■ ■ CH(a,){ri) 



3. Compute the rank of M over F 2 . Accept if the rank is less than q; reject 
otherwise. 



Proposition 2. There is a coRP- algorithm for Hypercube-Lin running in 
time 0{q^ + q^ns log s) . 



Proof. Fix any SC [g]. It follows from linearity and Lemma 1 that 

^iL(a*) = 0 if and only if ^ = 0 V r G {0, (1) 

ieS ieS 



Therefore, if H{ai ), . . . , H{aq) are linearly dependent, there is some S for which 
(1) holds. Then, the rows of M identified by S add up to 0, and thus M has rank 
less than q. On the other hand, if H{a\), . . . , H{aq) are not linearly dependent, 
then for all S C [g], ^H(ai) is not the zero codeword^, and thus 



Pr 




CH(a.)(o) = 0 Vj = l,2...,^ 




e 



Now, the probability that M has rank less than q is equal to the probability that 
there is some S C [< 7 ] for which (1) holds, which by a union bound is at most 
2'?-(2/3)^ < 1/2. Finally, computing M takes time 0{q£ns\ogs)^ and performing 
Gaussian elimination on M takes time 0{q^£), which yields a total running time 
of 0{q^ + q'^nslogs). 



^ In fact, 5/igs CH{ai) is not necessarily a codeword in the code defined in Lemma 2, 
but it is still a codeword in the code obtained by concatenating a Reed Muller code 
with a Hadamard code. Hence, the analysis for the relative distance of the code also 
shows that 5/igs has relative distance at least 1/3 from the zero codeword. 
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3.2 A Deterministic Algorithm for Hypercube-Lin 

The randomized algorithm for hypercube linearity testing is based on Lemma 1, 
which allows us to reduce testing a particular dependency between hypercubes 
to a polynomial identity test. In the actual algorithm, the power of randomness 
is used twice. First, randomness allows us to efficiently perform the polynomial 
identity test from Lemma 1. Second, using standard amplification of success 
probabilities we can take a union bound over the exponentially many possible 
linear dependencies between hypercubes in the correctness proof for the random- 
ized algorithm. 

Even though polynomial identity testing remains notoriously hard to deran- 
domize, Raz and Shpilka [5] recently found a deterministic algorithm that works 
for certain alternate and restricted models of computation. In particular, their 
algorithm works for formulas over arbitrary fields in non-commuting variables 
(i.e., formulas in the noncommutative ring F{a;i, . . . , a;„}), and for EUS circuits 
- depth three multilinear arithmetic circuits with a plus gate at the root. 

The formal polynomial pi + . . . +pm in Lemma 1 can be trivially computed by 
a,iu EUE circuit. This immediately gives a deterministic way of testing whether 
a particular linear relation Hi + . . .+Hm = 0 is satisfied over F 2 . However, we are 
interested in testing whether any one of the exponentially many relations ciHi + 
... -I- Cm Hm = 0 holds, or equivalently, if there exists coefficients ci , . . . , Cm G F 2 , 
not all zero, such that c\pi + . . ■ + CmPm = 0 as a polynomial in W 2 [xi,yi]. We will 
show that a construction, along the lines of Raz and Shpilka’s algorithm, works 
for this problem as well. Instead of trying to come up with a general theorem for 
testing linear dependencies between polynomials, we will focus on polynomials 
representing hypercubes, though it may be possible to extend the analysis to a 
somewhat more general scenario. 

Let F now be an arbitrary finite field, and pi, ■ ■ ■ ,Pm be polynomials in F[xij], 
where i G [n],jG [m] and each pk,k G [m] has the following form: 

n m 

Pk{xij) = 'y ' a^^Xjj , ( 2 ) 

i=ij=i 

where a^j G F are constants. It is easy to check that the polynomials in Lemma 1 
are written in this form. We are interested in determining, in time polynomial 
in m and n, whether there exist coefficients ci , . . . , Cm G F such that cipi -|- . . . -|- 
CmPm is the zero polynomial over F. Since this is a multilinear polynomial, this 
is equivalent to asking whether pi, . . . , pm are linearly dependent as elements 
of the vector space V over F generated by the monomials Xij^ . . -Xnj^^, where 
ji, . . . , j„ range over [to]. 

As in [5], the analysis works by induction on n. When n = 1, we have 
Pk{x\j) = '^J^iCtijXij, so the vectors pk G V are independent if and only if 
the matrix M G with M[j,k] = a\j has full rank. Using Gaussian elimi- 

nation this can be checked in, say, 0(TO^log|F|) time. 

We now show how to reduce a problem of degree n to one of degree n — 1 . 
The trick is to look at the following partial expansion of the polynomials pk- 
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m 

Pk{Xij) = 

ii,i2=i 



where qk is given by 



n m 

9fc(^(3...n)j) qk{x^j ^ X^j ^ ^ Xdj') ^ ^ O^jjXjj. 

i=3j=l 

Now consider a linear combination P{xij) = Yl'k=i^kPk{xij). We can expand 
this as 

m m 

P{xij) = XI • Cfcgfc(a;(3,„„)j). 

iij2 = l k=l 

From this expansion, we see that P = 0 in F[a;ij] if and only if for all ji, j 2 € [m]: 

m 

X ■ Ckqk{x^ 3 ,,,n)j) = 0 in F[a;( 3 ,„„)j]. (3) 

fe=i 

Consider the matrix M £ F™ whose rows are indexed by pairs (ji,j 2 ) G 
[m] X [to] such that M[(ji,j 2 ), k] = ceij^Oi 2 j 2 - Choose a subset S' of to rows such 
that the rows indexed by S span the row space of M. Then the constraints (3) 
are all satisfied if and only if 

m 

For all s £ S, X M[s, k] ■ Ckqk{x( 3 ...n)j) = 0 in F[a;( 3 ,..„)^-]. (4) 

fe=i 

We now want to rewrite the set of constraints (4) as a single constraint. For this, 
we introduce to additional formal variables i/s with s £ S. Constraints (4) are 
satisfied if and only if 

m 

X^^X M[s,k]- Ckqk{x( 3 ,„n)j) =0 in F[ys,a;( 3 .„„)j]. (5) 

sGS k—1 

Finally, let 

Xk{Vs,X(3„,n)j) = (^^^^^M[s,k]ys'^qkix(^3,„„)j), 

and note that constraint (5) is satisfied if and only if X^fcLi CkXkiVs, a^( 3 ...n)j) = 0 
in F[j/s,a ;(3 „)j]. To summarize: 

Lemma 3. The polynomials p\, . . . ,pm are linearly independent in F[a;y] if and 
only if the polynomials ri, . . . , are linearly independent in F[t/s, a ;(3 „)j] 

® Moreover, the pi satisfy a particular dependency X] CiPi = 0 if and only if the ri 
satisfy the same dependency. This can be used to find linear dependencies between 
hypercubes, which makes the decision to search reduction in the proof of Proposi- 
tion 1 unnecessary. 
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Now we can apply the induction hypothesis to the polynomials Vk, which have 
the prescribed form (2). The bottleneck in the running time of the algorithm is 
computing the linearly independent set S', which can be done in 0(m^ log |F|) 
time by Gaussian elimination. This yields a total running time of 0{m^nlog |F|), 
and concludes the proof of Theorem 1 . 

4 Extensions 

4.1 A Connection with Quantum Computation 

The following problem was brought to our attention by Aaronson in his recent 
work addressing skepticism of quantum computing [1]. The construction below 
implies the existence of explicit ( const ructible in deterministic polynomial-time) 
quantum states of provably superpolynomial tree size. The argument is similar 
to the proof of Proposition 2. 

Proposition 3. For every S < 1/4, there is a polynomial-time constructible 
k X n binary matrix M with k = , such that a random k x k submatrix of M 

has rank at least k — 1 with constant probability. 

Aaronson’s original argument requires M to have full rank with constant 
probability, but it is not difficult to see that his argument applies even if M has 
only rank k — 1 with constant probability. 

Proof. Fix e > 0 and choose ko such that k = fco log . Consider the concate- 
nation of a Reed-Solomon code with codeword size (over smallest possible 
alphabet size) with a Hadamard code. The resulting binary code C has length 
at most n = and relative distance 1/2 — 77 , where rj = Pick a 

basis ei, . . . , Cfe of {0, 1}^ and take M to be the k x n matrix whose zth row is 
the codeword C{ei). 

Now consider choosing a random k x k submatrix of M . Since k = 0(n^/^), 
with probability 1 — o(l), this is equivalent to choosing an independent set of k 
columns Mi, . . . , Mk of M with repetition. For any nonzero vector x G {0, 1}^, 
the probability that all dot products x'^ Mi , . . . , x'^ Mk vanish is at most ( 1/2 -|- 
77 )^. It follows that the expected number of nonzero x for which all these dot 
products vanish is at most 2^ • (1/2-1- 77 )^ < exp(2fc“'^) = 1-1- o(l). By Markov’s 
inequality, the number of such x is less than two with probability at least, say, 
1/3. If this is the case, then the columns of M satisfy at most one linear relation, 
so that the rank of M is at most fc — 1 . 

4.2 Towards a Stateless Implementation? 

We discuss our conjecture regarding a possible separation between stateful and 
stateless perfect implementations in the random oracle model. We say that a set 

5 of vectors in F 2 admits an efficient basis B where R is a (standard) basis for 
F| if every vector v G S can be written as the sum of poly(n) vectors in B and 
the basis representation can be computed in poly(r 7 ) time. 
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Proposition 4. If S admits an efficient basis, then there is a perfect (stateless) 
implementation of a random function f : {0, 1}" ^ {0, 1}, and answers queries 
that succinctly describe a set S, taken from a specific class of sets, with the value 

0o.es /(^)- 

It is easy to see that intervals of {0, 1}” admit an efficient basis, namely the 
collection of vectors {vi € F| ,i = 1,2, . . . , 2"} where Vi is the vector whose first 
i positions are ones and whose remaining positions are zeros. This observation 
underlies our construction for implementing random functions support interval- 
XOR queries in Section 1.2. On the other hand, we do not know if the hypercubes 
of {0, 1}" admit an efficient basis. 

Acknowledgements 

We thank Luca Trevisan and the anonymous referees for comments on an earlier 
version of this paper. 



References 

1. Scott Aaronson. Multilinear formulas and skepticism of quantum computing. In 
Proceedings of the 36th ACM Symposium on Theory of Computing, 2004. 

2. Oded Goldreich, Shafi Goldwasser, and Silvio Micali. How to construct random 
functions. Journal of the ACM, 33(4):210-217, 1986. 

3. Oded Goldreich, Shafi Goldwasser, and Asaf Nussboim. On the implementation of 
huge random objects. In Proceedings of the 44th IEEE Symposium on Eoundations 
of Computer Science, pages 68-79, 2003. Preliminary full version at 

http : //www. wisdom, weizmaim. ac . il/~oded/p_toro .html. 

4. Michael Luby and Charles Rackoff. How to construct pseudorandom permutations 
from pseudoranom functions. SICOMP, 17:373-386, 1988. 

5. Ran Raz and Amir Shpilka. Deterministic polynomial identity testing in non commu- 
tative models. In Proceedings of the 1 7th Conference on Computational Complexity, 
2004. To appear. 

6. J. T. Schwartz. Fast probabilistic algorithms for verification of polynomial identities. 
Journal of the ACM, 27(4):701-717, 1980. 

7. R. E. Zippel. Probabilistic algorithms for sparse polynomials. In Proceedings of 
EUROSAM 79, pages 216-226, 1979. 




Strong Refutation Heuristics for Random fc-SAT 



Amin Coja-Oghlan^, Andreas Goerdt^, and Andre Lanka^ 

^ Humboldt-Universitat zu Berlin, Institut fiir Informatik 
Unter den Linden 6, 10099 Berlin, Germany 
CO j a@inf ormatik . hu-berlin . de 
^ Technische Universitat Chemnitz, Fakultat fiir Informatik 
Strafie der Nationen 62, 09107 Chemnitz, Germany 
{goerdt , lanka}@inf ormat ik . tu-chemnitz . de 



Abstract. A simple hrst moment argument shows that in a randomly 
chosen fc-SAT formula with m clauses over n boolean variables, the frac- 
tion of satisfiable clauses is at most 1 — 2~^ + o(l) as m/n oo almost 
surely. In this paper, we deal with the corresponding algorithmic strong 
refutation problem-, given a random fc-SAT formula, can we find a cer- 
tificate that the fraction of satisfiable clauses is at most 1 — 2~^ o(l) 
in polynomial time? We present heuristics based on spectral techniques 
that in the case k = 3, m > ln(n)®n®^^ and in the case k = 4, m > Cn^ 
find such certificates almost surely, where C deuotes a constant. Our 
methods also apply to some hypergraph problems. 



1 Introduction and Results 

The k-SAT problem - given a set of fc-clauses, i.e. disjunctions of k literals 
over a set of boolean variables, decide whether there exists an assignment of 
the variables that satisfies all clauses - is the generic NP-complete problem. In 
addition to the decision version, the optimization version MAX fc-SAT ~ given a 
set of fc-clauses, find an assignment that satisfies the maximum number of clauses 
- is of fundamental interest as well. However, Hastad [18] has shown that there 
is no polynomial time algorithm that approximates MAX fc-SAT within a factor 
better than 1 — 2“^, unless V = NV. Hence, it is NP-hard to distinguish between 
instances of fc-SAT in which a (1 — e)-fraction of the clauses can be satisfied, and 
instances in which every truth assignment satisfies at most a (1 — 2“^ -|- £r)-share 
of the clauses for any £ > 0. Indeed, Hastad’s NP-hardness result is best possible, 
as by picking a random assignment, we can satisfy a (1 — 2“^)-fraction of the 
clauses in polynomial time. 

These hardness results motivate the study of heuristics for fc-SAT or MAX 
fc-SAT that are successful at least on a large class of instances. From this point 
of view, the satisfiability problem is interesting in two respects. First, one could 
ask for heuristics for finding a satisfying assignment (in the case of fc-SAT) or a 
“good” assignment (in the case of MAX fc-SAT). This problem has been studied 
e.g. by Flaxman [11], who has shown that in a rather general model of random 
satisfiable formulas a satisfying assignment can be found in polynomial time al- 
most surely (cf. also [22] for an extension to semirandom formulas). Secondly, 



K. Jansen et al. (Eds.): APPROX and RANDOM 2004, LNCS 3122, pp. 310-321, 2004. 
@ Springer- Verlag Berlin Heidelberg 2004 




Strong Refutation Heuristics for Random fc-SAT 



311 



one can ask for heuristics that can refute a fc-SAT instance, i.e. find a certificate 
that no satisfying assignment exists; of course, in the worst-case this problem 
is coNP-complete. In this paper, we deal with the second problem. More pre- 
cisely, we present strong refutation heuristics, i.e. heuristics that certify that no 
assignment satisfying considerably more than the trivial (1 — 2“^)-fraction of the 
clauses exists. One motivation for studying this problem is the relationship be- 
tween the existence of strong refutation heuristics and approximation complexity 
pointed out by Feige [8] . 

In order to analyze a heuristic rigorously, we need to specify on which type of 
instances the heuristic is supposed to work properly. In this paper, we consider 
a standard model of random instances of MAX k-SAT. Let V = {x\, . . . ,Xn} 
be a set of n boolean variables. Then, there are (2n)^ possible fc-clauses over 
the variables V. If 0 < p < 1, then we let Form„^fc^p be a random set of k- 
clauses obtained by including each of the (2n)^ possible clauses with probability 
p independently. Hence, the expected number of clauses in Form„_fc_p is m = 
{2n)^p. 

The combinatorial structure of random fc-SAT formulas has attracted con- 
siderable attention. Friedgut [12] has shown that Form„ ^ p exhibits a sharp 
threshold behavior-, there exist numbers Cfe = Cfe(n) such that Form„^fc^p is satis- 
fiable almost surely if m < (1 — e)ckn, whereas Form„^fc^p is unsatisfiable almost 
surely if m > (1-1- e)ckn. The asymptotic behavior of Cfc as fc ^ oo has been 
determined by Achlioptas and Peres [3] (building on the work of Achlioptas and 
Moore [1]). Moreover, a simple first moment argument shows that the maximum 
number of clauses of Form„^fc^p that can be satisfied by any assignment is at 
most (1 — 2^ -I- o(l))m as m/n oo. More precise results have been obtained 
by Achlioptas, Naor, and Peres [2]. 

Various types of resolution proofs for the non-existence of satisfying as- 
signments have been investigated on Form„_fc_p. Ben-Sasson [5] has shown that 
tree-like resolution proofs have size exp(f7(n/A^/^^“^)+^)) almost surely , where 
A = n’^~^p and 0 < £ < 1/2 is an arbitrary constant. Hence, tree-like resolution 
proofs are of exponential length even if the expected number of clauses is 
(i.e. p = 1/n^^^) for any constant <5 > 0. Furthermore, [5, Theorem 2.24] shows 
that general resolution proofs almost surely have exponential size if p < 

(d > 0 constant). 

Goerdt and Krivelevich [17] have suggested a heuristic that uses spectral 
techniques for refuting Form„q_p with p = ln(n)^n“^ (i.e. the expected number 
of clauses is m = In(n)^n^). No efficient resolution-based refutation heuristic 
is known for this range of p. Removing the poly logarithmic factor, Feige and 
Ofek [9] and (independently) Coja-Oghlan, Goerdt, Lanka, and Schadlich [7] 
have shown that spectral techniques can be used to refute Form„q_p if p > Cn~^ 
for a sufficiently large constant C > 0. Moreover, Feige and Ofek [10] have shown 
that a heuristic that combines spectral techniques with extracting and refuting 
a XOR formula from Form„^ 3 _p can refute Form„^ 3 _p for p > (i.e. m = 

This result improves on previous work by Friedman and Goerdt [13], 
and Goerdt and Lanka [16]. We emphasize that in all of the above cases, the 
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values of p to which the refutation heuristics apply exceed the threshold when 
Form„.fc_p becomes unsatisfiable almost surely by at least a factor of 

The new aspect in the present paper is that we deal with strong refutation 
heuristics. That is, our aim are heuristics that on input Form„^fc^p almost surely 
certify that not more than a (1 — 2“^ + e)-fraction of the clauses can be satisfied, 
for any £ > 0. This aspect has not (at least not explicitly) been studied in 
the aforementioned references. For instance, resolution proofs cannot provide 
strong refutation. Moreover, the spectral heuristics studied so far [7, 9, 10, 13, 
16,17] only certify that every assignment leaves a o(l)-fraction of the clauses 
unsatisfied. With respect to MAX 3-SAT, we have the following result. 

Theorem 1. Suppose that p > ln(n)®n“^/^. Let e > 0 be an arbitrarily small 
constant. There is a polynomial time algorithm 3 -Refute that satisfies the fol- 
lowing. 

— Correctness: For any MAX 3-SAT instance p, the output of 3-Ref ute{ip) is 
an upper bound on the number of satisfiable clauses. 

— Completeness: If p = Form„^ 3 _p, then 3-Refute{p) < (7 + s)n^p almost 
surely. 

Since the number of clauses of Form„. 3 _p is ~ 8n^p almost surely, 3-Refute 
does indeed certify almost surely that not more than a | + £ fraction of the 
clauses can be satisfied. Note that the value of p required for Theorem 1 is by a 
factor of ln(n)® larger than that required by the heuristic of Feige and Ofek [10] 
(which does not provide strong refutation). We sketch the proof of Theorem 1 
in Section 3. The following result addresses MAX 4-SAT. 

Theorem 2. Suppose that p > con~^ for a suffciently large constant cq > 0. 
There is a polynomial time algorithm 4-Refute that satisfies the following. 

— Correctness: For any MAX 4-SAT instance p, the output of 4-Ref ute{p) is 
an upper bound on the number of satisfiable clauses. 

— Completeness: If p = Form„q_p, then almost surely 4-Ref ute{p) < 15n^p + 
cin^ytp, where ci > 0 is a constant. 

4-Refute almost surely certifies that not more than a + 0{{ny/p)~^) 
fraction of the clauses can be satisfied. The second order term 0{{ny/p)~^) gets 
arbitrarily small as n^p grows. Theorem 2 applies to the same range of p as the 
best previously known refutation heuristics [7, 9] for 4-SAT, but provides strong 
refutation. We describe the heuristic for Theorem 2 in Section 2. 

The algorithms for Theorems 1 and 2 build on and extend the techniques 
proposed in [7,15]. For instance, 4-Refute constructs several graphs from the 
input formula p = Form„^ 4 _p. To each of these graphs, 4-Refute applies a sub- 
routine that tries to certify that the graph has “low discrepancy”; i.e. every set 
of vertices spans approximately the expected number of edges. This subroutine 
in turn relies on computing the eigenvalues of a certain auxiliary matrix. Finally, 
if all graphs have passed the discrepancy check, then we conclude that the input 
formula p does not admit an assignment that satisfies more than 15n'^p-|-cin^y^ 
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clauses. The MAX 3-SAT algorithm for Theorem 1 proceeds similarly, but is a 
bit more involved. Though in contrast to [7, 15] we obtain strong refutation 
heuristics, the algorithms and the proofs in the present paper are simpler. 

We point out that the approach in [15] can be used to obtain a heuristic 
that almost surely certifies that at most a (| + £)-fraction of the clauses of 
Form„^ 3 _p can be satisfied, though this issue is not addressed explicitly in that 
paper. However, the methods in [15] only seem to apply to somewhat bigger 
values of the clause probability p (namely, p > , i5 > 0 fixed) than those 

addressed in Theorem 1. 

The techniques that the algorithms 3-Refute and 4-Refute rely on yield 
heuristics for a variety of further hard computational problems, e.g. for hyper- 
graph problems. Recall that a k-uniform hypergraph H consists of a set V{H) 
of vertices and a set if (iJ) of edges. The edges are subsets oiV{H) of cardinal- 
ity k. An independent set in ii is a set S C V{H) such that there is no edge 
e G E{H) with e C S. The independenee number a{H) is the number of ver- 
tices in a maximum independent set. Moreover, H is called K-colorable, if there 
exists K independent sets Si,. .. ,3^ in H such that S'! U • • • U S'k = R (H). The 
chromatic number x{H) is the least integer /t > 1 such that H is ^-colorable. 

In analogy with the Form„_fc_p model of random fc-SAT instances, there is 
the ii„,fc,p-model of random fc-uniform hypergraphs: the vertex set of Hnx,p is 
V = {1, . . . , n}, and each of the (^) possible edges is present with probability 0 < 
p < 1 independently. Krivelevich and Sudakov [21] have solved the combinatorial 
problem of determining the probable value of the independence number and of 
the chromatic number of random hypergraphs. The following two theorems deal 
with the algorithmic problem of refuting that a 3-uniform hypergraph has a large 
independent set, or that a 4-uniform hypergraph is K-colorable. We sketch the 
heuristics for Theorems 3 and 4 in Section 4. 

Theorem 3. Let e > 0 be arbitrarily small but fixed. Suppose that p = f jr?!'^ , 
where In® n < / = o(n^/^). There is a polynomial time algorithm 5-Ref uteind 
that satisfies the following. 

— Correctness: If H is a 3-uniform hypergraph, then 3-RefuteInd(H) either 
outputs “a is small” or “fail”. If the answer is “a is small”, then a{H) < en. 

— Completeness: On input H = Hn, 3 ,p, 3-RefuteInd(H) outputs “a is small” 
almost surely. 

Theorem 4. Let k > 2 be an integer. Suppose that p > CQK^n~^ for some suffi- 
ciently large constant cq > 0. There is a polynomial time algorithm 4-RefuteCol 
that satisfies the following. 

— Correctness: If H is a 4,-uniform hypergraph, A-RefuteCol{H) either outputs 
“not K-colorable” or “fail”. If the answer is “not n-colorable” , then x{H) > 

K. 

— Completeness: On input H = Hn,i,p, the output of A-RefuteCol{H) is “not 
K-colorable” almost surely. 
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2 Random MAX 4-SAT 

In Section 2.2 we present the heuristic for Theorems 2. The main tool is a 
procedure for certifying that a random bipartite graph is of low discrepancy. 
This procedure is the content of Section 2.1. 



2.1 Discrepancy in Random Bipartite Graphs 

Throughout, we let Vi = {ui, . . . , v„} and V 2 = {wi, . . . , w„} be two disjoint 
sets consisting of n labeled vertices each. We consider bipartite graphs G with 
bipartition (Fi, V 2 ), i-e. the vertex set of G is Vi U V 2 , and all edges of G have 
one vertex in Vi, and one in V 2 . If Si C Vi and S 2 C then we let Eq{S\, S 2 ) 
denote the set of edges in G that connect a vertex in with a vertex in S 2 - 
Furthermore, Bn^p denotes a random bipartite graph obtained by including each 
possible edge {vi,Wj} with probability p independently. The aim in this section 
is to prove the following proposition. 

Proposition 5. Suppose that np > cq for some sufficiently large constant cq > 

0. There is a polynomial time algorithm BipDisc and a constant ci > 0 such 
that the following two conditions hold. 

1. Let G he a bipartite graph with bipartition (Vi,V 2 ). On input G, BipDisc 
either outputs “low discrepancy” or “fail”. If BipDisc{G) outputs “low dis- 
crepancy”, then for any two sets Si CVi, i = 1, 2, we have 

||-S'i||5'2|p - \Eb{Si,S 2 )\\ < Cl V|S'i||S' 2 |np + nexp(-np/ci). (1) 

2. BipDisc{Bn,p) outputs “low discrepancy” almost surely. 

If |S'i|, IS '21 = f2(n), then Eq. (1) entails that the number |Eg(S'i, £' 2)1 of edges 
from £1 to £2 in G deviates from its expectation |£i||£ 2 |p “not too much”. The 
crucial point is that BipDisc certifies that Eq. (1) holds for all sets £ 1 , £ 2 . 

BipDisc is based on computing the eigenvalues of a certain auxiliary matrix. 
Given a graph B with bipartition (Vi, V 2 )) we let A = A{B) = {aij)ij=i^,,,^„ be 
the matrix with entries = 1 if {vi,Wj} G E{B), and aij = 0 if {vi,Wj} ^ 
E{B). Let J denote an n x n matrix with all entries equal to 1. Then, we let 
M = M{B) = pj — A{B). Furthermore, let ||M|| = sup{||M^|| : ^ G R", ||5|| = 
1} denote the norm of M. On input B, ||M|| can be computed in polynomial 
time up to an arbitrarily small additive error. The next lemma shows what ||M|| 
has to do with discrepancy certification. 

Lemma 6. Let B he a graph with bipartition (Vi,V 2 )- Then, for any two sets 
Si cVi, i = 1,2, the inequality \ |Eb(£i,£ 2 )| - |£i||£ 2 b| < \/|£i||£ 2 | • \\M{B)\\ 
holds. 

Sketch of proof. Let £,i be the characteristic vector of Si, i.e. the j’th entry of 
(respectively ^ 2 ) is 1 if vj G £1 (respectively wj G £ 2 ), and 0 otherwise. 
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Then ||^i|| = Hence, |(M^2,‘Ci)l < \/|*S'i||S'2| • \\M\\. Moreover, a direct 

computation shows that = |5'i||S'2|p— S'2)|. □ 

In the case np > ln(n)^/n, one can show that ||M|| < 0{^/rvp) almost 
surely (via the “trace method” from [14]). Hence, in this case, by Lemma 6 
we could certify that (1) holds almost surely just by computing ||M(H„_p)||. In 
the case np = 0(1), however, we almost surely have that ||M(i3„^p)|| np, 

i.e. ||M(i3„^p)|| is much too large to give the bound (1) (cf. [20] for a detailed 
discussion) . Following an idea of Alon and Kahale [4] , we avoid this problem by 
removing all edges that are incident with vertices whose degree is too high (at 
least lOnp, say). This leads to the following algorithm. 

Algorithm 7. BipDisc(G) 

Input: A bipartite graph G = {Vi,V2,E). 

Output: Either “low discrepancy” or “fail” . 

1. If the number of vertices in G that have degree > lOnp is > nexp(— C2np), 
then output “fail” and halt. Here C2 > 0 is a sufficiently small constant. 

2. If the number of edges in G that are incident with vertices of degree > lOnp 
is larger than C3n^pexp{—C2np), where C3 > 0 is a sufficiently large constant, 
then halt with output “fail” . 

3. Let G' be the graph obtained from G by deleting all edges that are incident 
with vertices of degree > lOnp. Let M = M{G'). If j|Mjj > C4y/np for a 
certain constant C4, then output “fail” and halt. 

4. Output “G has low discrepancy” . 

The analysis of BipDisc is based on two lemmas. 

Lemma 8. There are constants C2, C3 > 0 such that almost surely B = Bn^p has 
the following properties. 

1. The set S of all vertices of degree > lOnp has cardinality < nexp(— C2np). 

2. The number of edges in B that are incident with at least one vertex in S is 
< csn^p exp{—C2np) . 

Lemma 9. There is a constant C4 > 0 such that almost surely the random 
bipartite graph B = Bn^p enjoys the following property. Let B' be the graph 
obtained from B by deleting all edges that are incident with vertices of degree 
> lOnp in B. Then, j|M(H')jj < a^Tip. 

Lemma 8 follows from a standard computation. The proof of Lemma 9 is 
based on estimates on the eigenvalues of random matrices from [4] . 

Proof of Proposition 5. Let G = Bn^p, and let Si C Vi for i = 1,2. Moreover, 
let S be the set of vertices of degree > lOnp in G. Suppose that BipDisc(G) 
answers “low discrepancy”. Then, by Lemma 6, 



I Eg{Si \S,S2\S)\- |5i \ ,S||^2 \S\p\< C4V\Si\\S2\np. 
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Moreover, because of Step 2 of BipDisc, we have |i?G(«5'i, S' 2 )| — \Eg{Si \ S', S '2 \ 
S)| < C 3 n^pexp{—C 2 np). Further, 

l>S'i||S 2 |p- |Si \ SIIS 2 \ S|p < 3np\S\ < nexp(-c2np/2), 

as otherwise Step 1 would have failed. Thus, (1) holds for Si, S 2 . Finally, by 
Lemmas 8 and 9, BipDisc(i3„^p) outputs “low discrepancy” almost surely. □ 

2.2 The Refutation Heuristic for 4-SAT 

Throughout this section, we let V = {xi, . . . , a;„} be a set of n propositional 
variables. Moreover, we assume that n?p > co for a large constant cq. 



Table 1. Clause types and unsatisfied clauses in the case of 4-SAT. 
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Let (/? be a set of 4-clauses over V. To employ the procedure BipDisc from 
Section 2.1, we construct 16 bipartite graphs , • ■ • , G^^®^ from p. Each G^*^ is 
a graph with bipartition (Ei, V 2 ), where Vi = V xV x{z} (i.e. Vi,V 2 are disjoint 
copies of M xV). Each graph G^*^ corresponds to one of the 16 possible ways to 
place the negation signs in a 4-clause: in G^*) , the edge { {xi^ {xi ^ , Xi ^ , 2) } 

is present iff the clause V V V is contained in ip, where is either 
Xi^ or Xij, according to the negation signs in Table 1. For instance, the edge 
{{xi-^,Xi^, 1), (xij, 2)} is in iff the clause V Xi^ V Xi^ V Xi^ occurs in 
ip. Thus, each clause of p induces an edge in one of the graphs G*^*\ and each 
edge results from a unique clause. The algorithm for Theorem 2 is as follows. 

Algorithm 10. 4-Refute(<^) 

Input: A set p of 4-clauses over V. 

Output: An upper bound on the number of satisfiable clauses. 

1. If the number of clauses in p is larger than 16n^p + n^^/p, then return the 
total number of clauses in p as an upper bound and halt. 

2. Compute the graphs G^*) and run BipDisc(G^*^) for i = 1, . . . , 16. If the 
answer of BipDisc(G(*^) is “fail” for at least one i, then return the total 
number of clauses in p and halt. 

3. Return 15n^p-|- cin^^/p, where ci is a sufficiently large consant. 
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Let us first prove that 4-Refute outputs an upper bound on the number of 
clauses that can be satisfied. 

Lemma 11. There is a constant C 2 > 0 such that the following holds. Let (p he 
a set of 4:- clauses such that BipDisc{G^^^) answers “low discrepancy” for all i. 
Then there is no assignment that satisfies more than \ip\ —n'^p+C 2 n^y/p clauses. 

Proof. Consider an assignment that sets the variables T C V to true, and F = 
R \ T to false. We shall bound the number of edges in the graphs that 
correspond to unsatisfied clauses. Let Ai C Vi and Bi C V 2 be the sets defined 
in Table 1 for z = 1, . . . , 16. Then, in the graph G^*\ the edges corresponding to 
unsatisfied clauses are precisely the Ai-Bi-edges. Thus, invoking Proposition 5, 
we have the following bound on the number of unsatisfied clauses: 

16 4 y s 

\EGi{A„B,)\ > X! ( j > n'^p- C2U^y/p, 

i=l i=0 ^ ' 

where C 2 ,cs are suitable constants. □ 

Proof of Theorem 2. The correctness follows from Lemma 11. Since by Chernoff 
bounds (cf. [19, p. 26]) the number of clauses in Form„^ 4 _p is at most 16n^p + 
o{n^y/p) almost surely, the completeness follows from Proposition 5. □ 

3 Random MAX 3-SAT 

While our refutation heuristic for 4-SAT is based on certifying that certain (bi- 
partite) graphs are of low discrepancy, the heuristic for 3-SAT needs to certify 
that a couple of triple systems are of low discrepancy. In Section 3.1, we de- 
scribe the procedure for certifying low discrepancy in triple systems. Then, in 
Section 3.2, we show how to employ this procedure in order to refute MAX 
3-SAT strongly. 

3.1 Discrepancy in Triple Systems 

Let V = {xi , . . . , Xn} be a fixed set of cardinality n. In this section, we consider 
triple systems over V, i.e. subsets S' C P x P x P. If Pi, P2, P3 C P, then we let 
(Pi)P 2 ,p 3 )s signify the set of triples (ui,z; 2 ,z; 3 ) € S with Vi G p for i = 1,2,3. 
Let e > 0 be a constant. We say that S has low discrepancy with respect to e if 
the following holds for all A C P with en < |A| < (1 — e)n: letting P = P \ A 
and a = |A|/n, we have 

|(A,A,A)s| = (l + o(l))-a3.|^|, 

|(A,A,P)s|,|(A,P,A)s|,|(P,A,A)s| = (l + o(l))-a2(i_c,) . | 5 |, 

|(A,P,P)s|,|(P,A,P)|,|(P,P,A)s| = (l + o(l))-a(l-a)2-|5|, 

|(P,P,P)s| = (l + o(l))-(l-a)3.|S|. 
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For 0 < p < 1, we obtain the random triple system Sn,p by including each 
triple in with probability p independently. The aim of this section is to prove 
the following propostion. 

Proposition 12. Let e: > 0. There is a polynomial time algorithm TripleDisc^ 
that satisfies the following conditions. 

~ For each triple system S C the output of TripleDisc^(S) is either “low 
discrepancy” or “fail”. If the output is “low discrepancy” , then S has low 
discrepancy w.r.t. s. 

~ If P ^ ln(n)®n“^/^, then the output of TripleDisc^{Sn,p) is “low discrep- 
ancy” almost surely. 

To certify that the triple system S' C is of low discrepancy, the algorithm 
TripleDisc constructs three projection graphs Gij, 1 < i < j < 3. The vertex set 
of Gij is V, and the edge {x, y} is present in Gij iff there is a triple {zi,Z 2 , 23 ) G S 
with X = Zi and y = Zj, or x = Zj and y = Zi. Thus, if S = Sn,p, then the edge 
{x, y} is present in Gij with probability p' ~ 2np independently of all other 
edges, so that Gij is distributed as a binomial random graph Gny . 

We say that a graph G = {V, E) has low discrepancy w.r.t. e if for all X C V 
of cardinality en < \X\ < (1 — e)n we have 

\\EG{X)\-\X\^n-^\E\\<e\E\A\\EG{X,V\X)\-2\X\{n-\X\)n-^\E\\<e\E\, 

where Eg{X) is the set of edges in G with both vertices in X, and Eg{X, Y) is 
the set of edges in G with one vertex in X and the other in Y. One ingredient 
to the algorithm TripleDisc for Proposition 12 is to certify that the graphs Gij 
are of low discrepancy. The following lemma provides us with a polynomial time 
algorithm for this problem. 

Lemma 13. Let £ > 0. Suppose that p' > There is a polynomial time 

algorithm A that satisfies the following conditions. 

— Correctness: For any graph G = (V,E), the output of A{G) is either “low 
discrepancy” or “fail”. If the output is “low discrepancy” , then G has low 
discrepancy w.r.t. e. 

— Completeness: If G = Gny, then the output of A{G) is “low discrepancy” 
almost surely. 

The proof of Lemma 13 is based on the relationship between graph discrepancy 
and eigenvalues (cf. [ 6 ]) and results on the eigenvalues of random symmetric 
matrices [14]. 

In order to certify that the triple system S has low discrepancy, it is, how- 
ever, not sufficient to check that the projection graphs Gij are of low discrepancy. 
Therefore, in addition to the projection graphs, one could consider the product 
graph Gt^ = {V x V,Et^), which is defined as follows: an edge {(ai, 6 i), ( 02 , 62 )} 
is in Et^ iff there exists a z € V such that there are two different triples 
(oi, 02 , 2 ), ( 61 , 62 , z) G S. Note that in contrast to the projection graphs Gij, 
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the product graph G,r is not distributed as a binomial random graph (the edges 
do not occur independently). If the projection graphs Gy and the product graph 
Gtt all have low discrepancy, then S is of low discrepancy as well. 

However, for the values of p in Proposition 12, we do not know a direct way 
to derive bounds on the eigenvalues of the adjacency matrix of the product graph 
(e.g. it seems difficult to apply the methods in [4,9,15]). Therefore, instead of 
dealing with the product graph and its adjacency matrix, we consider the matrix 
A = A{S,p) defined as follows. For 0 < p < 1 and &i, 62 , z G P we let Bh^h^z = 
Bb^b 2 z{S,p) = -1 if (&i, 62 ,z) G S, and Bb^b 2 z = Bb^b 2 z{S,p) = p/(l - p), 
otherwise. Then, the x n^-matrix A = A{S,p) = (afcici,& 2 C 2 )( 6 i,ci),( 62 ,c 2 )ev 2 
is given by 

afei 

Ci,& 2 C 2 — E {Bb^b 2 z ' Bq-^q^z + Bb^biz ' Bq^ciz) if (^ 1 , ^ 2 ) (ci, C 2 ), 

zev 

and abici,b 2 C 2 = 0 if ( 61 , 62 ) = (ci,C 2 ). Since A is symmetric and real-valued, the 
matrix has real eigenvalues Ai > • • • > A„ 2 . We let ||A|j = max{Ai,— A„ 2 } 
signify the norm of A. 

If S' C V^, X G V, and i G {1,2,3}, then the degree of x in slot i is dx,i = 
|{(zi,Z 2 ,Z 3 ) G S : Zi = xjj. We say that S is asymptotically regular if dx,i = 
(1 -I- o(l))n“^|S| for all x,i- Equipped with these definitions, we can state the 
following sufficient condition for S being of low discrepancy. 

Lemma 14. Let / = and suppose that In® n < / = o(n^/^). If S is a 

triple system that satisfies the following four conditions, then S is of low dis- 
crepancy w.r.t. £ > 0. 

1. s= jSj = . (1 + 0 ( 1 )). 

2. S is asymptotically regular. 

3. The three projection graphs of S are of low discrepancy with respect to e > 0. 
4- ITe 6 a?;e II A(S,p)|| < In® n • /. 

As by Lemma 13 we can check in polynomial time whether the conditions in 
Lemma 14 hold, we obtain the following algorithm. 

Algorithm 15. TripDisCg(S) 

Input: A set S C P®. Output: Either “low discrepancy” or “fail”. 

1. Check whether Conditions 1-4 in Lemma 14 hold. 

2. If so, output “low discrepancy”. If not, return “fail”. 

In order to prove Proposition 12, it remains to establish that the algorithm 
is complete. A standard application of Chernoff bounds (cf. [19, p. 26]) shows 
that the random triple system S = Sn,p with p as in Proposition 12 satisfies 
Conditions 1-2 in Lemma 14 almost surely. Moreover, the Condition 3 holds 
almost surely by Lemma 13. Thus, it suffices to show that Condition 4 holds 
almost surely. The rather technical proof of the following lemma is based on the 
trace method from [14]. 

Lemma 16. Let f > ln(n)®, and let p = fn~^^^. If S = Sn,p, then ||A(5',p)|| < 
In® n ■ f almost surely. 
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3.2 The Refutation Heuristic for 3-SAT 

Let be a set of 3-clauses over the variable set V = {xi, . . . ,Xn}- To ap- 
ply the procedure TripDisc from Section 3.1, we construct 8 triple systems 
C T® from ip, each corresponding to one of the 8 possible ways 
to set the negation signs in a 3-clause. In the triple system the triple 
(xij^ , Xi^ ,Xi^) G is present iff the clause V V occurs in ip, where 
either li. = Xi^ or U. = Xi^ , according to the negation signs for the clause types 
(where clause types are defined similarly as in Section 2.2). 

Algorithm 17. 3-Ref ute(<^, e) 

Input: A set p of 3-clauses over V. 

Output: An upper bound on the number of satisfiable clauses. 

1. Compute the triple systems and run TripDisCg^g (S'W) fori = 1,...,8. 
If the output is “fail” for at least one i, then return the total number of 
clauses in p as an upper bound and halt. 

2. Return (7 -I- e)n^p. 

The proof of Theorem 1 relies on a similar argument as the proof of Lemma 11. 

4 Hypergraph Problems 

Let H = (V, E) = Hn,A,p be a random 4-uniform hypergraph with vertex set V = 
{1, . . . , n}. Let K be an integer, and suppose that p > for a sufficiently 

large constant cq. The algorithm 4-RefuteCol for Theorem 4 is randomized. On 
input H, the algorithm obtains a set S' C of ordered 4-tuples as follows (recall 
that the edges E are not ordered). If e = {xi,X 2 , x^, X 4 } G E, then there are 4! = 
24 possibilities to order the vertices Xi,X 2 ,X 3 ,X 4 . Let T(e) be the set of the 24 
possible ordered tuples. Letting pg = 1 — (1 — we choose the set 0 yf Ae C 

T(e) of tuples that we include into S to represent e according to the distribution 
P(Ae) = Pq^‘'( 1— po)^'^“l''''lp“^. Thus, each edge e G E gives rise to at least one 
tuple in S. The choice of the sets Xe is independent for all e G E. Furthermore, 
we include each tuple (xi,X 2 ,X 3 ,X 4 ) G such that \{xi,X 2 :X^^X 4 }\ < 4 into S 
with probability po independently. A trite computation shows that if iJ = 
then the resulting set S = S{H) of 4-tuples is distributed so that every possible 
4-tuple in is present with probability pg independently. 

Let Vi =V xV X {1}, V 2 = V xV X {2} be two disjoint copies of V. Having 
computed S = S{H), 4-Ref uteCol constructs a graph G with bipartition (Vi , V 2 ) 
in which the edge {{x\,X 2 , 1), (xg, X 4 , 2)} is present iff {xl,X 2 ,xz^ X 4 ) G S.lf H = 
Hn, 4 ,p, then G is a random bipartite graph To this graph G, 4-Ref uteCol 

applies the procedure BipDisc. If BipDisc answers “low discrepancy”, then 4- 
Ref UteCol answers “H is not K-colorable” . Otherwise, the output is “fail”. 

The heuristic 3-RefuteInd(iL) for Theorem 3 transforms the hypergraph H 
into a triple system and applies TripleDisc. 
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Abstract. It is exponentially unlikely that a sparse random graph or 
hypergraph is connected, but such graphs occur commonly as the giant 
components of larger random graphs. This simple observation allows us 
to estimate the number of connected graphs, and more generally the num- 
ber of connected d-uniform hypergraphs, on n vertices with m = 0{n) 
edges. We also estimate the probability that a binomial random hyper- 
graph Hdin,p) is connected, and determine the expected number of edges 
of Hd{n,p) conditioned on its being connected. This generalizes prior 
work of Bender, Canfield, and McKay [2] on the number of connected 
graphs; however, our approach relies on elementary probabilistic meth- 
ods, extending an approach of O’Connell, rather than using powerful 
tools from enumerative combinatorics. We also estimate the probability 
for each t that, given k — 0{n) balls in n bins, every bin is occupied by 
at least t balls. 



1 Introduction and Results 

A d-uniform hypergraph H consists of a set V of vertices and a set E of edges, 
which are subsets of V of cardinality d. Thus, a 2-uniform hypergraph is just a 
graph. A vertex w is reachable in H from a vertex v if either v = w or there is 
a sequence ei, . . . , of edges such that v G ei, w € Ck, and Cj fl Cj+i yf 0 for 
i = 1, . . . , k—l. Reachability in H is an equivalence relation, and the equivalence 
classes are the connected components of H. We say that H is connected if there 
is only one component. 

Connectedness is perhaps the most basic property of graphs and hypergraphs, 
and estimating the number of connected graphs or hypergraphs with a given 
number of vertices and edges is a fundamental combinatorial problem. In most 
applications, one is interested in asymptotic results, where the number of ver- 
tices/edges tends to infinity. The main result of this paper is a formula for the 
asymptotic number of connected d-uniform hypergraphs; or, equivalently, an es- 
timate of the probability that a random d-uniform hypergraph is connected. 

* Research supported by the Deutsche Forschungsgemeinschaft (DFG FOR 413/1-1). 
** Supported by NSF PHY-0200909 and Los Alamos National Laboratory. 
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Our results hold up to a constant multiplicative factor. We study both a model 
Hd{n,m) in which the number of vertices and edges is fixed, and a binomial 
model Hd{n,p) in which each possible edge appears with probability p; in the 
latter case we also calculate the expected number of edges conditioned on the 
graph being connected. Furthermore, we obtain a simple algorithm for generating 
a connected hyper graph uniformly at random. 

We obtain these results using a new probabilistic approach. Rather than using 
powerful techniques of enumerative combinatorics such as generating functions 
and complex analysis, our calculations are for the most part elementary, and 
rely on the fact that a connected (hyper)graph of a given order and size is 
likely to occur as the giant component of a larger one. We believe that this 
approach is of interest in its own right, and can be applied to a number of 
problems in combinatorics and random (hyper)graph theory; to illustrate this, 
we also calculate the probability that every bin in a balls-and-bins experiment 
is occupied by at least t balls. 

For the special case of graphs, i.e., for d = 2, the results we present in this 
paper (and, in some cases, stronger results) are already known. Compared to 
graphs, little is known about d-uniform hypergraphs where d > 3, although 
several papers deal with their component structure (cf. Section 2 for details). 

Given a hypergraph H = (V, E), its order is its number of vertices \V\, and 
its size is its number of edges \E\. The degree dniv) of a vertex r; G R is the 
number of edges e G E such that v G e. Note that if El is d-uniform, its average 
degree is d\E\/n, Throughout, V = {1, . . . , n} is a fixed set of n labeled vertices 
and d > 2 is a fixed integer. We let Cd{n,m) signify the number of connected 
d-uniform hypergraphs of order n and size m. Observe that if a hypergraph is 
connected, then its average degree is at least d/(d — 1). 

Two natural random models present themselves. In Hd(n,m), we select one 
of the (y) sets of m possible edges at random. In Hd{n,p), each of the ())) edges 
appears independently with probability p, in which case the expected number 
of edges is p(^) ~ pn'^/d\ and the expected degree is ~ pn'^~^ /{d — 1)!. 

Thus we will compare Hd{n,m) where m = cn/d with Hd{n,p) where p = 
c{d — l)!/n‘^“^. While these two models are interchangeable in many respects, 
with respect to large deviations they are not; in particular, as we will see, their 
probabilities of connectedness differ by an exponential factor. 

The following theorem determines the asymptotic probability of connect- 
edness of Hd{n,m), or equivalently the number of connected d-uniform hy- 
pergraphs, up to a constant factor in the regime where the average degree is 
c > d/(d — 1). 

Theorem 1. Let c > dj (d— 1) be a constant independent ofn, and let m = cn/d. 
Let a = a(c) be the unique solution in (0, 1) of the equation 

l-(l-a)^-i\ 

1- (1-a)^ ; 



1 — a = exp 



—ca • 



( 1 ) 
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Fig. 1. Left figure: the functions 4>3{c) (black) and ^^(c) (gray) in the range 1.5 < c < 3. 
Right figure: the functions 73(c) (black) and c 1— » c (gray) in the range 0.5 < c < 5. 



and set d>d{c) = (1 — (1 — (1 — . Then the probability that 

Hd{n,m) is connected is 0(f?rf(c)"), and so Cd{n,m) = 0^^d(c)"(y)^ (o^ 
n ^ 00 ). 

Theorem 1 addresses the combinatorial problem of estimating the number of 
connected hyper graphs. But there is also a corresponding algorithmic problem: 
can we sample a connected hypergraph of a given order and size uniformly at 
random? We answer this question in the affirmative. 



Theorem 2. Let c > d/{d — 1), and let m = cn/d. There is a randomized 
algorithm that samples a connected hypergraph of order n and size m uniformly 
at random in expected polynomial time. 



Our next theorem gives an result analogous to Theorem 1 for Hd{n,p). 

Theorem 3. Let c > 0 be a constant, and let p = c{d — Let a be the 

unique solution in (0, 1) of the equation 



1 — a = exp 






( 2 ) 



and set Td{c) = a (1 — ^ exp ^ ° Jp ^ • Then the probability that 

Hd{n,p) is connected is 0(!f'd(c)"). For d = 2, >f'2(c) = 1 — exp(— c). 

To illustrate Theorems 1 and 3 we plot the functions ^a(c) and 1^3(0) in Fig- 
ure 1. In fact <^3 is larger than ^3, i.e., Hd(n,p) is exponentially more likely to be 
connected than Hd{n,m) even when the two have the same (expected) average 
degree. The reason for this is that in Hd(n,p) the number of edges is a random 
variable; we can think of Hd{n,p) as first choosing a number of edges m' ac- 
cording to the binomial distribution Bin((”),p), and then choosing a Hd{n,m'). 
Thus Hd{n,p) can boost its probability of being connected by including a larger 
number m' of edges. Indeed, if we condition on Hd{n,p) being connected, the 
conditional expectation of the number of edges will be significantly larger than 
p(^). Our next theorem quantifies this observation. 
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Theorem 4. As in Theorem 3, let c > 0 be a constant independent of n, let 
p = c{d — and let a be the unique solution in (0,1) of Equation (2). 

Set 



ld{c) = c 



1-a-aY 



The expected number of edges of Hd{n,p) conditioned on the event that Hd{n,p) 
is connected is n^d{c)/d ± o{n), so the expected average degree is ^d{c) ± o(l). 
For d=2, 72 (c) = c • (e'^ + l)/(e'^ — 1) = c • coth(c/2). 



The proof of Theorem 4 consists of maximizing the product of d>d(c = dm/n)'^ 
times the binomial distribution Bin(())),p). Figure 1 shows 73 (c). As c ^ 00 , 
7 d(c) ^ c, since connectedness becomes a less unusual condition as c increases. 
As c ^ 0, 7 c(d) ^ d/{d— 1), the minimum average degree required for connec- 
tivity. 

One ingredient of the proofs of Theorems 1-4 is the component structure of 
random hypergraphs Hd{n,m) and Hd{n,p). We say that Hd{n,m) or Hd{n,p) 
has a certain property with high probability (w.h.p.) if the probability that the 
property holds tends to 1 as n ^ 00 . Suppose that the average degree c = 
dm/n (resp. c = pn'^~^/{d — 1)! is a constant greater than l/{d — 1). Then, 
just as for graphs [ 6 ], w.h.p. there is a unique giant component of order f2{n) 
in Hd{n,p), and all other connected components are of order O(lnn) [7, 13]. To 
prove Theorems 1-4, it is crucial to have a rather tight estimate on the order 
and size of the giant component. 

We say that a random variable X defined on Hd{n, m) or Hd{n,p) is concen- 
trated in width y about x if for each e > 0 there is a number C (e) independent of 
n such that Pr[|A — x\> C{e)y\ < £. Further, we write f{n) = 0{g{n)) if there 
is a constant c > 0 such that |/(n)| < | 5 (n)| In'^ n for all sufficiently large n. 

Theorem 5. Suppose that c > l/{d— 1). Let a = a(c) be the unique solution in 
( 0 , 1 ) of the equation 

1 - a = exp (-C (1 - (1 - a)'^"^)) 

and let b = b{c) = 1 — (1 — a)^^. Let m = cn/d and let p = c{d — 

Then, the expected order (resp. size) of the giant component of Hd{n,m) and 
of Hd{n,p) is an -1-0(1) (resp. bm 0(1)). Moreover, the order (resp. size) of 
the giant component of both Hd{n,m) and Hd{n,p) is concentrated in width y/ri 
about an (resp. bm). 



Note that setting d = 2 in Theorem 5 recovers the classic result 1 — a = e~'^“ 
for random graphs [ 6 ] . To further illustrate the applicability of our approach, we 
solve a type of occupancy problem. Suppose k balls are placed randomly in n 
bins; the following theorem estimates the probability qt(k,n) that every bin is 
occupied by at least t balls. 



Theorem 6. Let t > 1 be an integer, let r > t be a constant independent of n, 
and let k = rn. Let a = a{r) and (3 = (3{r) be the unique solutions in (0, 1) to 
the pair of equations 
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3=0 3=0 



r- 



Set ri = r ■ and let A = A(r) he the unique solution of the equation 

(^-^ 1 )! = ^1 ^ unless t = 1, in which case set A = 1. Then set 



t-1 



l-l/a 



(j){r) = ^(1 — a) 






3=0 



where bj = 



i3\ 



Then, qt{k,n) 



0(</)(r)") as n ^ oo. 



For i = 1 in particular, Theorem 6 gives a = 1 — e~’’“, (3=1, and ri = 0, 
A = 1, and bo = 1. Then qi{k, n) = 0 ((o;^“’’e'’^““^^)"') . The proof of Theorem 6 
is similar to the proofs of Theorems 1 and 3 and is omitted. 



2 Related Work 

The asymptotic number of connected graphs with a given number of edges and 
vertices was determined by Bender, Canfield, and McKay [2]. Their proof is 
based on generating functions, and is stronger than Theorem 1 in that it de- 
termines the number of connected graphs up to a multiplicative factor 1 -|- o(l) 
instead of a constant. The same authors have determined the probability that 
a random graph Gn,p is connected as well as the expected number of edges of 
Gn,p conditioned on connectedness [3] , corresponding to the case d = 2 of The- 
orems 3 and 4; however, they give the asymptotic distribution of the number of 
edges conditioned on connectedness, not just its expectation. An algorithm for 
generating random connected graphs (the case d = 2 of Theorem 2) immediately 
follows from the results in [2, 3]. Pittel and Wormald [12] presented an alternative 
approach based on enumerating graphs of minimum degree 2 to derive slightly 
improved versions of the results in [2]. Among other things, O’Connell [11] has 
estimated the probability that a random graph Gn,p is connected using a prob- 
abilistic approach. However, the result in [11] is somewhat less precise than [3]. 
Using large-deviation methods from statistical physics, Engel, Monasson, and 
Hartmann [4] have investigated exponentially unlikely component structures of 
random graphs, including connectedness and, more generally, a specified number 
of components per vertex. 

The asymptotic order and size of the giant component of random graphs (the 
case d = 2 of Theorem 5) has been known since the pioneering work of Erdos 
and Renyi (cf. [6] for a unified treatment). Furthermore, Barraez, Boucheron, 
and Fernandez de la Vega [1] give a description of the asymptotic distribution 
of the order and size of the giant component of random graphs. 

For d-uniform hypergraphs, d > 3, on the other hand, little seems to be 
known. Karonski and Luczak [8] determined the number of connected hyper- 
graphs of order n and size m = n/{d — 1) -I- fc where k = o(lnn/lnlnn) (i.e.. 
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just above the number of edges necessary for connectedness) up to a factor of 
1 + 0 ( 1 ) via purely combinatorial techniques. Since Theorem 1 addresses the case 
m = cn/d for c > d/{d— 1), our results and those of [8] are incomparable. 

Schmidt-Pruzan and Shamir [13] have shown that in a very general model of 
random hypergraphs a phase transition phenomenon occurs: there is a certain 
average degree c* such that for all c < (1— e)c* the largest component of Hd{n, m) 
with m = cn/d has order O(lnn), whereas for c > (1 + e)c* there is a unique 
giant component of order f?(n). In the case of d-uniform hypergraphs, the critical 
average degree is c* = l/{d— 1). Karonski and Luczak [7] have studied this phase 
transition in greater detail. To the best of our knowledge, the expected order 
and size of the giant component as given in Theorem 5 have not been stated 
before (although our proof relies on standard methods). 

With regards to the occupancy problem, in the case t = 1, more precise results 
for qi(k,n), can be obtained from asymptotic formulas for Stirling numbers of 
the second kind due to Temme [14] and Moser and Wyman [10]. However, these 
results rely on generating functions and complex analysis, while our approach 
is elementary. For t > 1, to the best of our knowledge the results stated in 
Theorem 6 are new. 

3 Techniques and Outline 

While most of the previous work relies on rather heavy machinery from enumer- 
ative combinatorics, our approach is quite different and purely probabilistic, and 
yields shorter and simpler proofs. For instance, to prove Theorem 1 the basic 
idea is as follows. Suppose that to = cn/d for some c > l/(d — 1), and let a,b 
be as in Theorem 5. Then, by Theorem 5, w.h.p. there is a unique giant com- 
ponent in Hd{n,m), which — conditioned on its order and size — is a uniformly 
random connected hypergraph. As we shall see in Section 5, we can express the 
number Cd{an, bm) of connected hyper graphs of order an and size bm in terms 
of the probability x(an, bm) that the giant component of Hd{n, to) has precisely 
order an and size bm. Then, in Section 6, we reduce the problem of computing 
x(an, bm) to a balls-and-bins problem. Section 4 contains the proof of Theo- 
rem 5. The algorithm for Theorem 2 drops out of our proof of Theorem 1 in 
Section 6 immediately. 

Our approach is related to that of O’Connell [11], but goes beyond it consid- 
erably. O’Connell shows that Pr[G„,p is connected] = by observing 

that w.h.p. the giant component of Gn,p has order an + o{n) where 1 — a = e““'” 
(the d = 2 case of Theorem 5). However, due to the o(n) error term, in [11] it 
is necessary to prove that a certain rate function is continuous. This proof relies 
heavily on the independence of the edges in G„^p, and therefore is not easy to 
adapt to the G„,m model. By contrast, we do not need to work with rate func- 
tions, because we estimate the probability that the giant component of Gn,p has 
exactly order an. In effect, we determine Pr[G„_p is connected] up to a constant, 
rather than subexponential, factor. 
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4 The Giant Component of Hd{n,m) and Hd{n,p) 

To estimate the expected order and size of the giant component of a random hy- 
pergraph Hd{n,p), we establish an analogy between a Galton- Watson branching 
process with a suitable successor distribution and exploring the components of 
a random hypergraph. For graphs, a similar approach is well known (cf. [6]). To 
estimate the expected order/size of the giant component of a hypergraph up to 
an additive error of 0(1), we need to refine this approach slightly. 

Let us first describe the branching process. During the process there are two 
kinds of “organisms”: living and dead ones. In the beginning, there is only one 
organism, which is alive. In the i’th step, a living organism is chosen, produces 
a number Zi of children, and dies. The Zi’s are independent and is distributed 
as a multiple (d — 1) • Po(c) of a Poisson random variable with mean c. We say 
that the process dies out if at some step i the last living organism dies without 
producing any offspring. General results [5, p. 297] imply that the probability 
p that the process dies out is the unique solution in (0, 1) of the equation p = 
exp — 1)). 

Now, for random hypergraphs H = Hd(n,p), consider the following process, 
which explores the connected component of a vertex v. During the process, the 
vertices of H are either dead, alive, or neutral. Initially, only v is alive, and all 
other vertices are neutral. In each step, a living vertex w is chosen. We investigate 
all edges e of H that contain w, at least one neutral vertex, but no dead vertex 
(since a dead vertex indicates that that edge has already been explored). All 
neutral vertices contained in such edges are made live, and w dies. Let Z^ be 
the number of vertices made live from w. When there are no living vertices left, 
the set of dead vertices is precisely the connected component of v. 

Recall that a random variable X dominates Y if Pr[P < t] < Pr[AT < t] for 
all t. Let Bin(fV, p) denote a binomial random variable with parameters N and 
p. Then, on the one hand, Zyj is dominated by {d — l)Bin((()“^),p), since the 
degree of w in Hd{n,p) is distributed as Bin((()“^),p). On the other hand, Zyj 
dominates the following random variable Sk'- with probability 1 — 0(l/n), Sk has 
distribution (d — l)Bin(())“^),p), and with the remaining probability Sk = 0. 
Using these estimates, a similar argument as in [6, proof of Theorem 5.4] yields 
the following. 

Lemma 7. Let a, b he as in Theorem 5. The expected number of vertices outside 
the giant component is (1 — a)n-|- 0(1), and the expected number of edges outside 
the giant component is (1 — b)m + 0(1). 

Lemma 7 establishes the first part of Theorem 5. In order to show that the 
order and size of the giant component of Hd{n,p) are concentrated about their 
means, we show that both have variance 0(n), and apply Ghebyshev’s inequality. 
A similar approach works for Hd{n,m). 

5 The Number of Connected Hypergraphs 

Suppose that m = cn/d for some c > 1/ (d— 1) . Following the outline in Section 3, 
in this section we reduce the problem of counting connected hypergraphs to 
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estimating the probability that the giant component of Hd(ji,m) has a given 
order and size. Let a, b be as in Theorem 5, and set 






"V0v(”7)ve)y'. 



( 3 ) 



Let Cd{an, bm) = Cd{an, bm) be the probability that Hd{an, bm) is con- 

nected. Then, 'tp{an,bm) • Cd{an,bm) is the expected number of components of 
order an and size bm in Hd{n,m). Furthermore, if iL is a hypergraph, then we 
let Af{H) (resp. M{H)) denote the order (resp. size) of the component with 
the most vertices (resp. edges). Then letting x(x,y) be the probability that 
H = Hd(n,m) satisfies Af{H) = x and M{H) = y, we obtain 



x(an, bm) < ip{an, bm) ■ Cd{an, bm) (4) 

(for d = 2, a similar inequality has been derived in [9]). Furthermore, bm) ■ 
Cd{an,bm) ■Vr\N'{Hd{n — an,m—bm)) < an] is the probability that in iLd(n, m) 
there is precisely one component of order an and size bm. Hence, 



■^(a, b) ■ Cd{an, bm) • Fr[Af{Hd{n — an, m — bm)) < an] < x(a^j bm). (5) 

Let us first get rid of the term Fr[Af{Hd{n — an, m — bm)) < an]. 

Lemma 8. Pi[Af{Hd{n — an, m — bm)) < an] ~ 1. 

Lemma 8 follows from the analogy between random hypergraphs and branch- 
ing processes discussed in Section 4. Lemma 8, (4) and (5) yield 



x(an, bm) ~ y(an, bm) ■ Cd{an, bm) . 



( 6 ) 



Thus, we have reduced the problem of computing Cd{an, bm) to the problem of 
computing xifl'n,bm). The following lemma, which we prove in Section 6, does 
this within a multiplicative constant. 

Lemma 9. x{o-n,bm) = 0((nm)“^/^). 

Combining Lemma 9 and (6) gives Cd{an,bm) = 0 [{nm)~^^'^‘ip{an,bm)~^) . 
Let u = an and y, = bm. Expanding 'ijj(an,bm) using Stirling’s formula yields 






6>((nm)i/2) 



( 7 ) 



bm 



1 0 ( 1 -!»)"■ 
(•;)”>) I (“oo™ . 






Let = dy/v, i.e., the average degree of the giant component if it has order v 
and size y. Plugging the relation (1 — a)‘^ = 1 — b into (7), we obtain 

0{cd{n, y)) = (a^"‘^(l - a)’^ (l - (1 - . 




330 Amin Coja-Oghlan, Cristopher Moore, and Vishal Sanwalani 



Finally, using 1 — a = exp (— c (l — (1 — a)'^ ^)) and b = 1 — (1 — a)‘^ from 
Theorem 5 and writing c = dm/n = C,a/b, we get 

1 — a = exp 

thereby proving Theorem 1. 

To prove Theorem 3, we write an equation analogous to (3). If Cd{n,p) is the 
probability that Hd{n,p) is connected, then the expected number of connected 
components of order x is (”)(1 — ) • Cd{x,p) . We also have the 

following lemma, analogous to Lemma 9. 

Lemma 10. Let a be as in Theorem 5. The giant component of Hd{n,p) has 
order an with probability 

Theorem 3 then follows from Lemma 10 and algebraic manipulations similar to 
those for Theorem 1. 

To prove Theorem 2, we observe that Lemma 9 yields a very simple random- 
ized algorithm for generating a uniformly random connected hypergraph of order 
1 / and size p = fv/d for constant C > d/{d — 1) in expected polynomial time. 
First, fix n,m,c such that a{c)n = v and b{c)m = p where a = a(c), b = b{c) 
are as in Theorem 5. Then construct a random hypergraph Hd{n,m) and find 
its giant component C. Since C is uniformly random conditioned on its order and 
size, we output it if it has order n and size p, and if not we repeat. By Lemma 9, 
this simple rejection sampling approach succeeds in expected time polynomial. 



d-l 



-Ca 



1 -(!-«) 

1 - (1 - ay 



6 The Probability of Getting a Giant Component 
of a Given Order and Size 

Lemma 9 is a rather uncommon statement about random hypergraphs, since it 
concerns the probability that the order and size of the giant component attain 
exactly one particular value. A more common type of result (which is, of course, 
interesting in its own right) would be to characterize the asymptotic distribu- 
tion of e.g. Af{Gn,m)- For instance, in [1] it is shown that N{Gn,m)/ \/n has an 
asymptotically normal distribution. However, as this statement allows for addi- 
tive errors of size o(y/n), it does not imply Lemma 9. In this section, we shall 
prove that the following lemma on fluctuations in the classical balls-and-bins- 
problem implies Lemma 9. (Lemma 10 can be derived analogously.) 

Lemma 11. Suppose that we throw r balls into n bins uniformly at random and 
independently, where r = pn for some constant p > 0. Let Z denote the number 
of empty bins. Then for each G > 0 there is a S > 0 such that for all z satisfying 
\z — E(Z)| < C-y/n we have Fr[Z = z] > 

There are (at least) two ways to prove Lemma 11. On the one hand, one can 
derive the lemma from rather involved combinatorial results such as [14]. On 
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the other hand, there is also a simple probabilistic argument using sums of 
independent random variables (details omitted). 

Here is the basic idea for how to employ Lemma 11 to prove Lemma 9 . We 
expose the edges of the random hypergraph Hd{n,m) in two stages. First, we 
throw in mi = (1 — s)m random edges, thereby obtaining Hi = Hd{n, mi). Here 
e > 0 is a sufficiently small constant. Let C denote the giant component of Hi. 
Then, we obtain Hd{n, m) by adding a set F of m2 = em further random edges 
to Hi . Let be the set of all edges in F that have d— 1 vertices in C and whose 
d’th vertex either is an isolated vertex of Hi or belongs to an isolated edge of Hi. 
Then, we first add the edges in F\F]^qI\ to Hi. Finally, we add the edges in 
thereby attaching isolated vertices and edges to the giant component. We shall 
model the number of isolated edges/ vertices that we attach as a balls-and-bins 
experiment, the edges in being the balls, and the isolated edges/ vertices of 

Hi being the bins. By Theorem 5 , N{Hd{n, m) = Hi + F) and M{Hi + F) are 
concentrated in width about their means an, bm. Thus, in order to achieve 
e.g. that the order of Hi +F is precisely an, we need that the number of vertices 
attached to the giant component via is precisely an — N{Hi + F \ 
and indeed Lemma 11 estimates the probability that this is the case. We need to 
consider both isolated vertices and edges, because we wish to prove a statement 
on both the order and the size of the giant component. 

However, there are some details that we have glossed over so far. For instance, 
there could be edges in F that fall into H \ C and “eat up” isolated vertices 
that the edges in would like to use as bins. Moreover, probably there will 
be edges in F that contain vertices in the giant component of Hi and several 
isolated vertices, etc. Therefore, we need to consider a finer partition of F. First, 
we add those edges that either land inside the giant component of Hi or inside 
its complement entirely: let 

Fi = {e G F| e C C}, F2 = {e G F| e C H \ C}, H2 = Fi + Fi + F2 . 

If e is small enough, then w.h.p. all components of H2—C will have order O(lnn), 
so that C is still the giant component of H2. As a next step, we add 

— the set F3 of edges in F that join a component of order > doi H2—C to C, 

~ the set F4 of all edges e G F that are incident with less than d — 1 vertices 

in C, 

— and the set F5 of edges e G F that share precisely d —1 vertices with another 
edge e yf e' G F. 

All edges in F3 U F4 U F5 join vertices in the giant component C of H2 with 
vertices in F \ C. Let F3 = F2 + F3 + F4 + F5, and let C be the vertex set 
of the giant component of F3. Furthermore, let F' be the set of all edges in 
e G F \ (Fi U • • • U F5) that join a component of order 1 or of order d (i.e. an 
isolated edge) of F3 to C'. By the choice of F4 and F5, F' has the following two 
properties. 

i. Each e G F' has d —1 vertices in C C C' . 

ii. For any two edges e, e' G F' we have e n C yf e' n C. 
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The edges in F' will be our “balls” . Finally, let Fq = F\{FiU- ■ - UF^UF'), and 
H4 = Ffs + Fq. Then, the edges in Fq connect C with an isolated vertex/edge of 
i/2, but we can’t use these edges as “balls”, because their isolated vertex/edge 
is also connected with C through another edge in F. Observe that all vertices of 
edges in Fq lie inside the giant component of Fl^, so that the vertex set of the 
giant component of i?4 is still C . 

Let 3^1 (resp. 3^2) be the set of isolated vertices (resp. edges) of H4, and set 
Yi = Moreover, let F[ (resp. F^) be the set of all e € F' that connect an 
isolated vertex (resp. edge) of i?4 with C , and set Zi = ifF^. Finally, let Xi 
(resp. X2) be the number of isolated vertices (resp. edges) that the edges in F{ 
(resp. F2) connect with C' . The following observation is crucial for our argument. 

Proposition 12. If we condition on any specific outcome 0 / (Yi, I 2 , ^ 1 , -^ 2 ); 
then Xi,X2 are independent. Moreover, the conditional distribution of Xi is 
precisely the distribution of the number of non-empty bins when Zi balls are 
thrown into Yi bins uniformly at random and independently. 

Corollary 13. Let Hi{Zi) = E(Jfi | Zi = Zi). Conditioned on Z\ = zi and 
Z2 = Z2, both Xi and X2 are concentrated in width y/ri about pLi{zi), pL2{z2)- 

Sketch of proof. Condition on the event that Z\ = zi, Z2 = Z2, and Yi = yi, 
Y2 = 2/2- By the proposition, Xi,X2 are independent and distributed as in a balls- 
and-bins experiment. Therefore, a standard application of Azuma’s inequality 
(cf. [6, p. 37]) shows that both Xi and X2 are concentrated in width -y/n. □ 

As a next step we prove that the number of balls and bins is of the same 
order of magnitude as n. 

Lemma 14. With probability > 1 — we have Yi,Y2, Zi, Z2 = C{n). 

Observe that the number of vertices that the edges in F' attach to the giant 
component C' of H4 is X\ + dX2. Furthermore, the number of edges (including 
the ones in F') joined to C via F' is Z\F Z2F X2. 

Lemma 15. Let C > 0 be a sufficiently large constant. Let Z denote the set of 
all pairs (zi,Z2) such that the following holds: 

Pr[\JV{H4) — an — {pL\{z\) + d^2(^2))| < C^/n and (8) 

\M{H4) — bm— {zi + Z2 + ^2(^2))! < C\/n \ Zi = z\,Z2 = Z2] > 1/2 . 

Then Pr[(Zi,Z2) € Z] > 99/100. 

Sketch of proof . Let A > 0 be a sufficiently large constant. On the one hand, by 
Corollary 13, the probability that the number of isolated vertices or the number 
of isolated edges attached to C deviates by more than Xyd from the expectation 
/ri(Zi), ^2(^2) is less than 6. On the other hand, by Theorem 5, the order and 
the size of the giant component of Hd{n,m) are concentrated in width ^/n. □ 

Proof of Lemma 9 (sketch). To prove the lower bound C((nm)“^/^), condition on 
the following event Si: we have Yi, Y2, Z\, Z2 = 12{n) and the outcome (zi, Z2) of 
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(Zi, Z 2 ) is such that (8) holds. By Lemmas 14 and 15, Pr[£i] > 9/10. Moreover, 
by Lemma 15, the conditional probability that satisfies 

\M{H4)-an-{fii{zi)+dfi2{z2))\, \M{H4)-bm-{zi+Z2+fi2{z2))\ < C^/n (9) 

is at least 1/2. We additionally condition on the event £2 that (9) holds. Then, 
Proposition 12 and Lemma 11 entail 

Pr[Xi + (IX 2 = an — Af{H 4 ), Zi + Z 2 + X 2 = bm — Xi{H4)\£in £ 2 ] = 0{nm)~^ . 

Furthermore, Pr[£i H £ 2 ] > 9/20, thereby proving the lower bound claimed in 
Lemma 9. The (even simpler) proof of the upper bound is omitted. □ 
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Abstract. Given two independent weak random sources X, Y, with 
the same length t and min-entropies bx , by whose sum is greater than 
i + C(polylog(.t/e)), we construct a deterministic two-source extractor 
(aka “blender”) that extracts max{bx, by) + {bx + by — i — 41og(l/e)) 
bits which are e-close to uniform. In contrast, best previously published 
construction [4] extracted at most ^{bx +by — ^ — 2 log(l/e)) bits. Our 
main technical tool is a construction of a strong two-source extractor 
that extracts {bx + by — £) — 21og(l/e) bits which are e-close to being 
uniform and independent of one of the sources (aka “strong blender”), 
so that they can later be reused as a seed to a seeded extractor. Our 
strong two-source extractor construction improves the best previously 
published construction of such strong blenders [7] by a factor of 2, ap- 
plies to more sources X and Y , and is considerably simpler than the 
latter. Our methodology also unifies several of the previous two-source 
extractor constructions from the literature. 



1 Introduction 

Imperfect Randomness. Randomization has proved to be extremely useful 
and fundamental in many areas of computer science. Unfortunately, in many 
situations one does not have ideal sources of randomness, and therefore has to 
base a given application on imperfect sources of randomness. 

Among many imperfect sources considered so far, perhaps the most general 
and realistic source is the weak source [29,4]. The only thing guaranteed about 
a weak source is that no string (of some given length £) occurs with probability 
more than 2~^, where b is the so-called min-entropy of the source. We will call this 
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source 6)-weak. Unfortunately, handling such weak sources is often necessary 
in many applications, as it is typically hard to assume much structure on the 
source beside the fact that it contains some randomness. Thus, by now a universal 
goal in basing some application on imperfect sources is to make it work with weak 
sources. The most direct way of utilizing weak sources would be to extract nearly 
perfect randomness from such a source. Unfortunately, it is trivial to see [4] that 
no deterministic function can extract even one random bit from a weak source, 
as long as 6 < ^ — 1 (i.e., the source is not almost random to begin with). This 
observation leaves two possible options. First, one can try to use weak sources 
for a given application without an intermediate step of extracting randomness 
from it. Second, one can try designing probabilistic extractors, and later justify 
where and how one can obtain the additional randomness needed for extraction. 

Using a Single Weak Source. A big and successful line of research [26, 24,4, 
5, 29, 2] following the first approach showed that a single weak source is sufficient 
to simulate any probabilistic computation of decision or optimization problems 
(i.e., problems with a “correct” output which are potentially solved more effi- 
ciently using randomization; That is, all problems in BPP). Unfortunately, most 
of the methods in this area are not applicable for applications of randomness, 
where the randomness is needed by the application itself, and not mainly for 
the purposes of efficiency. One prime example of this is cryptography. For ex- 
ample, secret keys have to be random, and many cryptographic primitives (such 
as public-key encryption) must be probabilistic. The problem of basing a cryp- 
tographic protocol on a single weak random source has only been studied in the 
setting of information-theoretic symmetric-key cryptography. In this scenario, 
the shared secret key between the sender and the recipient is no longer random, 
but comes from a weak source. As a very negative result, Mclnnes and Pinkas [12] 
proved that one cannot securely encrypt even a single bit, even when using an 
“almost random” {£,£— l)-weak source. Thus, one cannot base symmetric-key 
encryption on weak sources. Dodis and Spencer [6] also consider the question of 
message authentication and show that one cannot (non-interactively) authenti- 
cate even one bit using {£,£/2)-weak source (this bound is tight as Maurer and 
Wolf [11] showed how to authenticate up to £j2 bits when b > £j2£). 

Using Several Weak Sources. Instead, we will assume that we have several 
weak sources, each one independent from all the other weak sources. Specifically, 
we will try to extract nearly ideal randomness from two weak sources. 

The question of extracting randomness from two or more independent ran- 
dom sources originated in the works of Santha and Vazirani [20,25,27]. Chor 
and Goldreich [4] were the first to consider general weak sources of equal block 
length; let us say that the sources X and Y are {£x,bx)-^eak and {£y, 6v)-weak, 
and for simplicity we also assume £x = £y = £■ It is known [7] that in this setting 
it is possible to extract nearly all the entropy (i.e., nearly bx + by — 21og(l/e) 
bits) from the two sources, where e is the required distance from the uniform 
distribution. However, best known explicit constructions achieve much weaker 
parameters. Specifically, for one bit extractors Chor and Goldreich showed that 
the inner product function works provided bx + by > £ + 21og(l/£). The 
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best multi-bit two-source extractor ([4], thm 14) is capable of extracting almost 
^{bx + by — £ — 2 log( 1/e) ) random bits, but [4] only show how to efficiently im- 
plement this extractor when bx + by > 1.75^. It was also known how to extract 
a non-constant bit (or a few bits) from two sources when bx = -l-o(l)), by = 

0(log(^)) [9, 1], and this method can also be used to extract unbiased bits from 
such sources. In a setting of more than two weak sources, Barak et al. [3] re- 
cently show how to extract randomness from (1/(5)*^^^^ independent (f, i5f)-weak 
sources (for any i5 > 0), but this improvement does not apply to the case of two 
sources considered in this work. 

Motivated by cryptographic applications, Dodis and Oliveira [7] recently con- 
sidered the problem of extracting random bits from two independent sources 
which are (essentially) independent from one of the two sources (called the 
“seed”). They termed this two-source extractor a strong blender, since it is a 
common generalization of a two-source extractor (which they simply termed 
“blender”) and a strong extractor [16], where the seed source is assumed to be 
public and truly uniform (and the objective of the latter is to make the seed as 
short as possible). They showed the existence of such strong blenders, whenever 
bx,by > log^-|-21og(l/£), which are capable of extracting {bx — 21og(l/£)) bits 
from X which are independent from the {£, 6y)-weak seed Y. On a constructive 
side, they showed that the constructions from [4] for two-source extractors (i.e., 
“blenders”) and the related work of Vazirani [27] for the so-called SV sources [20] 
do extend to give strong blenders. The best such construction in [7] (based on a 
similar construction from [4]) is capable of extracting \{h x + by — £—2\og{\/ e)) 
bits independent of one of the sources, but this is again efficient only when 
bx + by > 1.75£. 

To summarize, while randomness extraction from two independent sources 
is possible, the known constructions and parameters seem far from optimal. 
In contrast, there are many efficient constructions of one-source randomness 
extractors (with truly random seeds) with nearly optimal parameters (see [10, 17, 
21] and the references therein). Our work gives an improvement in the number of 
extracted bits, for the case bx + by > I, allowing for the first time the extraction 
of more than half the total randomness that is present in the sources. 

Our Results. We give a simple two-source extractor capable of extracting 
max(6x, by) -I- {bx + by — £ — A\og{l/ e)) bits whenever the sum of min-entropies 
is slightly greater than £: bx + by > £ + l7(polylog(^/£)). Our construction is 
based on two observations. First, a strong blender with good parameters can give 
a two-source extractor (i.e., a “regular blender”) with even better parameters, 
when combined with a seeded extractor (with good parameters). Specifically, if 
a strong blender extracts k > 0(polylog(^/£)) nearly random bits Z which are 
independent from the weak seed Y, we can now apply a seeded extractor to Y, 
using the just extracted Z as the seed (since Z is “long enough” to be used 
with state-of-the-art extractors such as [17]). This allows us to extract a total of 
{k+by—2 log(l/£)) bits, which could be very high. While this obvious observation 
would already improve the state-of-the-art in the problem of two-source extrac- 
tion, even when combined with the known strong blender constructions from 
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[7], our second observation is a much simpler (and better) construction of strong 
blenders. Specifically, we show how to extract (fejf + &v— 2 log(l/e)) bits which 
are independent from one of the sources. Thus our strong blender construction: 

(1) improves by a factor of two the best previously published construction of [7]; 

(2) is very simple and efficient without any limitations on bx and by (as long as 
bx + by > £+21og(l/e)); and (3) uses only trivial properties from linear algebra 
in its analysis. By choosing the initial “seed” source to be the one containing 
more min-entropy, we get our final two-source extractor. 

Our Strong Blender. In fact, we give the following general technique for 
constructing our strong blender. Assume we wish to extract k nearly random 
bits from X using F as a seed. Let Ai,...,Afc be some £ x £ matrices over 
GF[2], specified later. View X and Y as f-bit vectors, and output bits {{AiX) ■ 
Y,...,{AkX) ■ F), where • denotes the inner product modulo 2, and AiX is 
the matrix- vector multiplication over GF[2], Of course, the main question is 
which condition on matrices Ai . . . A^ would allow us to maximize the value of 
k. As we show, all we need is to have every non-empty sum Ag = 

(where S' yf 0) of these matrices to have “high” rank. In particular, by using a 
known linear algebra construction [14,13,19]) of such matrices, we can ensure 
that all the needed matrices A$ will have full rank, which will in turn give us 
the desired value k = {bx + by — £ — 2log{l/ e)) . However, we also notice that by 
using several other simple, but sub-optimal choices of matrices Ai . . . A^, we will 
obtain two of the three constructions from [7] as special cases, as well as several 
other constructions which appeared in the literature for related problems (i.e., 
[27]). Thus, our methodology elegantly unifies several of the previous approaches 
into one elementary framework. 

2 Preliminaries 

2.1 Basic Notation 

We mostly employ standard notation. The symbol log is reserved for the base 2 
logarithm. For a positive integer t, Ut denotes a random variable that is uniform 
over {0, 1}* and independent of all other random variables under consideration. 
We also write [t] = {1,2,... t}. For two random variables A, B taking values in 
the finite set A, their statistical distance is |j A — B\\^ = i Xae.4 ' ~ 

Pr[B = a]!, and the min-entropy of A is Hoc, (A) = min^e^ ( — log(Pr[A = a])). 
For a random variable A taking values in (0, 1}*, the bias of A is its statistical 
distance from Ut, \\A — Ut\\g. The L 2 norm of a vector v G TZ^ is |lu |]2 = 

For two strings s,t, their concatenation is denoted (s , t). For a 
vector a € {0, 1}‘ and 1 < i < t, we say that i G a if at = 1. Also recall that 
an (^, 6)-source denotes some random variable X over {0, 1}^ with min-entropy 
Hoo (A) > b. 

2.2 Strong Blenders and Extractors 

We start by defining the notion of a randomness extractor [16]. 
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Definition 1 ([16]). Let b > 0, e > 0. A {b,e)~ extractor Ext : {0,1}^ x 
{0, 1}^ ^ {0, 1}™ is an efficient function such that for all {£,b)-sources Y, we 
have 

II Ext(Y, Uu) - C/n^ L < e 

Clearly, one of the main objectives in the area of building explicit extractors 
is to minimize the seed length k while still extracting nearly all the randomness 
from the source. Many nearly optimal constructions exist by now (see [15, 23, 10, 
17, 22, 21] and the references therein). While none of them clearly beats the rest 
in all parameters, it is known (e.g., [17]) how to extract m = k + b — 21og(l/e) 
nearly random bits (which is optimal) using a seed of length k = 0(polylog(^/e)). 

In this work, however, we are interested in (strong) randomness extraction 
from two weak sources. Correspondingly, we define the notion of two-source 
extractors (aka “blenders”) and strong blenders. For simplicity, we assume both 
sources have the same length £ throughout the paper. 

Definition 2. A {b x, by, s)- strong blender (SBJ is an efficient function Ble : 
{0, 1}^ X {0, 1}^ ^ {0, 1}^ such that for all {£, bx)-weak sources X and all {£, by)- 
weak sources Y , we have 

II {Y , ble(x,e)) - (y , Uk) |L<£ 

A {bx, by ,s) -two-source extractor is an efficient function Two-Ext : {0,1}^ x 
{0,1}^ ^ {0,1}™ such that for all {£,bx)-weak sources X and all {£,by)-weak 
sources Y, we have 



||Two-ExT(X,y) -UmW, < e 

We now observe the following relation among these three primitives which 
follows immediately from the triangle inequality: namely, given X, Y , the output 
of a strong blender Ble(X, Y) can be used as a random seed for an extractor 
Ext, to extract the randomness from Y. 

Lemma 1. Let Ble : {0, 1}^ x (0, 1}^ ^ (0, 1}^ be a {b x, by, e)- strong blender 
and Ext : {0, 1}^ x (0, 1}^ ^ (0, 1}™ he a {by ,s)- extractor. Then the following 
function Two-Ext : {0,1}^ x {0,1}^ ^ {0,1}™ is a {bx,by,2e)-two-source 
extractor: 

two-ext(x,y) = ext( y, BLE(y, y ) ) 

Our final two-source extractor will follow by applying the Lemma above 
to the strong blender which we construct in the next section (using any good 
seeded extractor, such as the one in [17]). Thus, from now on we concentrate on 
improved constructions of strong blenders. 



3 Efficient Strong Blender Constructions 

In this section we show a construction of an efficient strong blender. In 3.1 we 
give a general technique for constructing a strong blender, that requires a set 
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of £ X £ matrices over GF[2] with the property that the sum of every subset of 
matrices has high rank. In section 3.2 we give an explicit set of matrices (that 
appeared in [14, 13, 19]), with the property that the sum of any subset of matrices 
has full rank. This gives the best possible result for our technique. In the same 
section we also review some previous constructions for two-source randomness 
extraction as special cases of our technique. 

3.1 General Technique for Constructing Strong Blenders 

We now show how to extract many random bits from two weak sources X, Y of 
the same length £, such that the bits are random even if one sees Y. 

Let Ai, . . . , Akhe £x£ matrices over GF[2], such that for every nonempty subset 
SC [k], the rank of As Ai is at least f — r, for 0 < r < £ (we will want 

to have high rank and keep r as small as possible). 

The proposed strong blender is 

Ble^:{0,1}^x{0,1}^ ^ {0,1}'= 

(x,y) I — > i^(Aix) ■y,...,(Akx) -yj 

where • is the inner product mod 2 and AiX is a matrix- vector multiplication 
over GF\2], 

Theorem 1. The function Ble^i is a (&jf , 6v, e)-SB 
with log i = fex+fcy+ 2 -(f-Hr-Hfc) ^ 

II (y, blea{x,y)) - (y , Uk)\\^<2 ^ 

Our main tool for proving theorem 1 is the Parity lemma (also known as the 
XOR lemma, see [8,27]), which relates the bias of a fc-bit random variable T 
with the sum of squared biases of every subset of the bits in T. 

Lemma 2 (Parity lemma [8]). For any k-hit random variable T, ||T — Uk\\g 
is upper bounded by 

I E \\T-a-U,\\l 

where T ■ a is the inner product (mod 2) of T and a. 

We also use a relation between the rank of a matrix L and the min-entropy 
of LX. 

Lemma 3. Let L be an £ x £ matrix over GF[2]. If the rank of L is £ — r, then 
for any random variable X over {0, 1}^, Hoc, (LX) > Hoc (X) — r. 

Proof. Note that for any z G {0,1}^, there are at most 2'’ different values 
X G {0,1}^ such that Lx = z. Thus, Pr[LX = z] < 2’’ • 

max„,g{o,i}^ = a:]. 
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Proof, [of Theorem 1] Following [4,7], it is sufficient to consider the case when 
Y is uniformly distributed on some set S'y, jS'yj = 2^'^ , and X is uniformly 
distributed on some set Sx, j-Sxl = 2*'^. 

||(r , BLE^(x,y))-(y , u^)\\ = ^ ^ ||BLE^(x,y) - 

' veSv 

(by parity lemma) {\\BLEA{X,y)-a - C/i||J2 

yeSv ' y 0'=5^ae{0,l}'“ 







- 


(by concavity) < 


E 


E(l|BLEA(A,y)-a- C/ 1 IIJ 2 


\ 


0 *^5<iae{0,l}'= 



We now proceed to upper bound each of the (above) bracketed terms by 
2 -( 6 x+fcr+ 2 -^-r). final result then follows from simple addition. Fixing a 
value of a = aia2...ak G { 0 , !}^\{ 0 ^}, and identifying a with the set of all 
i G [k] for which = 1 , we have 



BLEA{X,y)-a = ^Blea(X,7/)j = 

i^a i^a 




where all sums are taken modulo 2. 

Now define Aa = (mod 2), and recall that the rank of Aa is at 

least £ — r. As a proof tool, we introduce the 2^ x 2^ matrix M whose rows 
and columns are labeled by the elements of {0, 1}^, and whose {x,y)th entry is 
Mx,y = (— (for x,y G {0,1}^). We also let be the xth row of M. 
The following two properties are the key to what follows. 

1. Every row of M is equal to at most 2’’ other rows of M. Indeed, if x,x' G 
{0, 1}^ satisfy = M^', then it must be that AaX = Aax' (mod 2), i.e. 
X + x' (mod 2) is in the kernel of Aa- Since the rank of Aa is at least £ — r, 
the kernel of Aa has at most 2’’ elements. 

2. Rows of M that are distinct are orthogonal (in the Euclidean inner product 

sense). For if a; yf x' G {0, 1}^ are such that f=- then z '=^ Aa{x + 
x') yf 0^ (mod 2), and the 77.- valued Euclidean inner product of and 
Mx' = {MxViMx') = Eyemvi-^y^ is 0. 

Now let D G be the vector defined by D 2|Sx| i® 

not hard to see that the yth coordinate Dy of D is equal to ^(Pr^; eSx Wx,y = 
l]-Brx(iSxWx,y = -1]), SO that \Dy\ = ||Ble^(A,i/) • a - Ui\\^. The quantity 
we wish to bound is thus 



— ^(||BLE^(A,y)-a- C/i||j2 



1 

I'S'vl 



E ^ 

y&Sv 



l^yj II II 2 
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and 



D 



equals the Euclidean inner product D"’" D over TZ 



D 






x,x'gSx 

But properties 1. and 2. above show that for any x € Sx 



x'eSx 






x' :M X 



< 2’' 



Mx 



< 2 



r+t 



Hence 

1 



D 



< 2'’+^/4|S'x|, and plugging this bound into (3.1) yields 



\Sy\ 



Y mL^A{X,y) ■ a - U^Wf < 






V^Sy 



4|S'x||5'y I 



^ 2-ibx+bY+2-(i+r)) 



which, as noted above, implies the theorem. 



We emphasize that the above technique is entirely general: any set of matrices 
Hi , . . . , Hfe such that the sum of any non-empty subset of matrices has rank of at 
least £ — r, can be used to extract k<bx + bY + 2 — £ — r bits from £-bit weak 
sources X, Y with respective min-entropies bx, by- 

The following two cases are of special interest (of course, they apply only 
when r is “small enough”; that is, when r + k < bx + by + 2 — £): 

1. If the min-entropies bx + by sum up to ^ -I- polylog(£), we can extract poly- 
logarithmic (in £) number of bits with bias 

2. If the min-entropies bx + by sum up to (I -I- c)£, for any constant 0 < c < I, 
we can extract linear (in £) number of bits with exponentially small bias. 



3.2 Explicit Strong Blender Instantiations 

Theorem 1 subsumes constructions introduced (sometimes implicitly) in pre- 
vious works. These constructions are presented below, followed by our more 
randomness-efficient construction. 

Identity Matrix. This is the simplest case when k = 1 and Ai is the identity 
matrix. This gives r = 0 and implies that the inner product function is a strong 

(b^ -f- 1 — ^ ^ 

blender with bias e = 2 2 , reproving the result from [7] (adapting the 

result from [4]). 

Cyclic Shift matrices. Vazirani [27] used cyclic shift matrices, to extract 
randomness from SV sources. When £ is a prime with 2 as a primitive root, the 
sum of any subset of matrices has rank at least £— 1 (that is r = 1, in our notation 
above). Formally, let Ai be a linear transformation matrix corresponding to a 
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cyclic shift by z — 1 bits; That is, the jth row of Ai has a 1 in the j — i + 1 
(mod £) column and zero elsewhere. The required property of these matrices is 
given below. 

Lemma 4 ([27]). Let £ he a prime with 2 a primitive root modulo £ (i.e. 2 is a 
generator of Zf). Let it G {0, 1}^ \ {0^, 1^} be a vector (which is not the all O’s 
or the all I’s vector). Let A he an £ x £ matrix over GF(2), such that the rows 
of A are the £ right- cyclic- shifts of u . Then rank{A) >£—l. 

By Theorem 1, a corresponding strong blender can extract k bits with bias 

( b -|- b -|- 1 — £ — fc) ^ / , s s 

2 2 , or, equivalently, extract k = {bx + 6f + 1— ^ — 2 log(l/£)) bits 

with bias e. 

(Non-Cyclic) Right Shift matrices. Let A\, . . . , Ak be linear transforma- 
tion matrices such that Ai corresponds to the right shift by z — 1 bits. For any 
non-empty subset S' C [k], it is easy to see that the rank of As = is 

at least £ — k -\- 1. By theorem 1 one can build a strong blender from those 

1 11 11 ^ — 2fc) 

matrices that extracts k bits with bias 2 2 , or, equivalently, extract 

k = {bx -\-by — £ — 21og(l/£r))/2 bits with bias e. 

The matrices from a general error-correcting code. The construction 
of [7] (adapted from [25]) also pertains to the present framework. Let C be 
an [£^k,d\ linear error correcting code (i.e., having dimension k and minimal 
distance d in {0, 1}^). Let Ai, . . . ,Ak be diagonal matrices, the diagonal of Ai 
containing the coordinates of a codeword Ci encoding the z-th unit vector in 
{0,1}^. By linearity, the diagonal of A$ = is also a codeword in C, 

which is non-zero when S yf 0, and thus has at least d ones, so that rank{As) > 
d. Then Theorem 1 applies with r = £ — d, and the corresponding strong blender 

(b 5^ d-by^ -]-ti — 2^ — fe) ^ 

extracts k bits with bias 2 2 ^ or, equivalently, extract k = [bx + 

by + d — 2£ — 21og(l/e)) bits with bias £ (notice, sometimes this k may not be 
achievable due to the coding constraints). 

Our construction. We now present our new construction of strong blenders 
via Theorem 1. We show that there is a set of matrices Ai, ... Ai whose non- 
empty sums have full rank, thus achieving the best possible result for using 
the technique in section 3.1. This latter fact follows from [14,13,19], and we 
reproduce its simple proof for completeness. We use the isomorphism {0, 1}^ « 
GF[2^]. Take any basis xi,X 2 , ■ ■ ■ ,X£ of GF[2^]. (There are many such bases for 
GF[2^], since it is isomorphic to the ^-dimensional vector space over GF[2]. One 
such basis, which is easy to use, is . . . ,x^~^ , where x is any primitive 

element of GF[2^]; that is, an element that is not the root of any polynomial of 
degree < £ over GF[2^].) The matrices {AijXi are defined such that matrix Ai 
corresponds to left-multiplication by the basis-element xu 

Al : y e GF[2^] ^ x^y 

Now for each non-empty set 5 C [^], xs = ^ non-zero element in 

GF[2^], and therefore has an inverse in GF[2^]. Let 17$ be the £ x £ matrix 
over GF\(2] that corresponds to left-multiplication by 
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Us -.y € GF[2^] ^ xg^y 

Now, it is easy to see that Us is the matrix-inverse of As = since 

left-multiplication by matrix Us'Yhi^s corresponds to multiplication by the 
field identity element ~ 

This immediately implies the desired conclusion: 

Lemma 5. Let Ai, . . . ,Ai be matrices presented above. Then for alUd ^ S C [P\, 
the rank of As 

Plugging the first k of the matrices Ai,...,Ai into theorem 1 we deduce 
the existence of an explicit strong blender that can extract k bits with bias 
2 2 , or, equivalently, extract k = {bx +bY + ‘2 — i — 2 log(l/£)) bits. 
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Abstract. The central observation of this paper is that if en random 
edges are added to any n-node connected graph or digraph then the 
resulting graph has diameter 0{\ogn) with high probability. We apply 
this to smoothed analysis of algorithms and property testing. 

Smoothed Analysis: Recognizing strongly connected digraphs is a ba- 
sic computational task in graph theory. Even for graphs with bounded 
out-degree, it is NL-complete. By XORing an arbitrary bounded out- 
degree digraph with a sparse random digraph R ~ we obtain a 

“smoothed” instance. We show that, with high probability, a log-space 
algorithm will correctly determine if a smoothed instance is strongly 
connected. We also show that if NL g almost-L then no heuristic can 
recognize similarly perturbed instances of (s, t)-connectivity. 

Property Testing: A digraph is called fc-linked if for every choice of 2k 
distinct vertices si, . . . , Sfc, ti, . . . , tfc, the graph contains k vertex disjoint 
paths joining Sr to tr for r = l,...,fc. Recognizing fc-linked digraphs 
is NP-complete for k > 2. We describe a polynomial time algorithm 
for bounded degree digraphs which accepts fc-linked graphs with high 
probability, and rejects all graphs which are at least en edges away from 
being fc-linked. 



1 Introduction 

The diameter of a graph G is the length of the longest shortest path in G. 
In other words, if d{u,v) is the length of the shortest path from rt to in G, 
then the diameter of G is max„ „ d{u,v). A graph is connected (and a directed 
graph is strongly connected) if it has finite diameter. The central observation 
of this paper is that if en random edges are added to any n-node connected 
graph then the diameter becomes G(logn) with high probability (meaning with 
probability tending to 1 as n ^ oo and abbreviated whp). This is also true for 
strongly connected directed graphs (digraphs) and for several ways of generating 
the random edges. For ease of exposition, we state this as a theorem only for a 
strongly connected digraph perturbed by adding a random digraph R ~ 
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Here i? ~ D means R is distributed according to distribution D, and V>n,p is the 
distribution over n-node digraphs where each arc is included independently with 
probability p. 

Theorem 1. Let e be any positive constant. For any strongly connected n-node 
digraph D, let D = D R where R ~ D„ Then whp the diameter of D is 
at most 100e“^logn. 

Similar results hold for perturbations formed by adding en edges selected at 
random with or without replacement, or by adding a random matching or as- 
signment with en edges, or by picking en vertices and adding a uniformly random 
out-arc to each of them. 

Theorem 1 is related to a class of problems regarding the possible change of 
diameter in a graph where edges are added or deleted, for example, the results of 
Alon, Gryarfas, and Ruszinko in [2] on the minimum number of edges that must 
be added to a graph to transform it into a graph of diameter at most d. The 
study of these extremal diameter alteration questions was initiated by Chung 
and Garey in [12]. It is also related to the theorem of Bollobas and Chung on 
the diameter of a cycle plus a random matching [10]. 

1.1 Application: Smoothed Analysis 

Recognizing strongly connected digraphs is a basic computational task, and the 
set of strongly connected digraphs is NL-complete [21]. Thus, if NL ^ L then 
there is no log-space algorithm which recognizes strongly connected digraphs. 

Perhaps this conclusion of worst-case complexity theory is too pessimistic. 
We will consider the performance of a simple heuristic which runs in randomized 
log-space. We will show that the heuristic succeeds on random instances whp. 
However, the “meaning” of this result depends on the probability space from 
which we draw the random instances. It seems reasonable to assume that any 
real-world digraph will contain some amount of randomness, so it is tempting to 
believe this result shows that in the real-world strong connectivity only requires 
log-space. Unfortunately, this is not valid if we use the “wrong” model for ran- 
domness. For example, the distribution V>n,p is pleasant for analysis, but basic 
statistics like degree sequence seem to differ from real-world graphs [14] . 

We will use a model of randomness that is more flexible. We will start with 
an arbitrary digraph D and perturb it by XORing it with a very sparse random 
graph R ~ This produces a random instance which is “less random” 

than Dn,p. The study of worst case instances with small random perturbations 
is called Smoothed Analysis. 

Smoothed Analysis was introduced by Spielman and Teng in [27] and they 
discuss a perturbation model for discrete problems in [28]. They consider per- 
turbing graphs by XORing the adjacency matrix with a random adjacency ma- 
trix, where each edge is flipped with some constant probability. Since the prob- 
ability of an edge flip is constant, the perturbed instances are all dense graphs 
(i.e. a constant fraction of all possible edges appear). Independently, Bohman, 
Frieze and Martin [7] studied the issue of Hamiltonicity in a dense graph when 
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random edges are added, and other graph properties were analyzed in this model 
by Bohman, Frieze, Krivelevich and Martin [8] and Krivelevich and Tetali [23]. 

We will also use an XOR perturbation, but we will make the probability of 
corruption much lower than [28]. Since we will have a linear number of edges 
present, it is appropriate for the perturbation to change about en edges, which 
is the expected number of edges in D„ j/„. 



Randomness and Strong Connectivity. Recognizing strongly connected di- 
graphs is closely related to recognizing (s, t)-connectivity in digraphs, which is 
the canonical NL-complete problem. It is possible to recognize connectivity in 
undirected graphs with a randomized log-space algorithm using random walks 
[1]. Since the cover time of an arbitrary connected graph is bounded by 0(ji^) 
[17], a random walk will visit every vertex in polynomial time whp. This ap- 
proach will not work for arbitrary digraphs, however, since there the cover time 
can be exponential. 

The diameter and connectivity of random graphs has been well-studied, see 
for example the books of Bollobas [9] and Janson, Luczak, and Rucihski [20]. 
Perhaps closest in spirit to our investigation is the paper of Bollobas and Chung 
on the diameter of a Hamilton cycle plus a random matching [10] and the paper 
of Chung and Carey on the diameter of altered graphs [12]. Also, the component 
structure of random digraphs was studied by Karp in [22] and more recently by 
Cooper and Frieze [13]. 



A Heuristic for Recognizing Strong Connectivity. For each ordered pair 
of vertices (s,t) repeat the following procedure Ni times: Starting from s, take 
N 2 steps in a random walk on the digraph. Here a random walk is the sequence 
of vertices Xq, Xi, . . . , Xt, . . . visited by a particle which moves as follows: If 
Xt = V then Xt+i is chosen uniformly at random from the out-neighbors of Xt- 
If the random walk with restarts ever reaches t, then the digraph contains an 
(s, t)-path, and we continue to the next pair of vertices. 

If Ni and N 2 are large enough, this algorithm is correct whp. For example, if 
there is a path from s to t and Ni = n^", N 2 = n we will discover the path whp. 
The values of Ni and N 2 can be significantly improved for smoothed random 
instances 

The main theorem of this section is that when Ni and N 2 are suitable poly- 
nomials in n, this heuristic, which we will call Algorithm A, is successful on per- 
turbations of bounded out-degree instances whp. To prove this, we first show 
it is successful when the initial instance is a strongly connected digraph and 
the perturbation only adds edges. Then we extend this to show success when 
the initial instance is not necessarily strongly connected and the perturbation 
only adds edges. After this, it is simple to translate our results to the original 
perturbation model where edges are added and removed, since we can gener- 
ate the perturbation in 2 rounds, by first deleting each existing edge with some 
probability, and then adding random edges to the resulting digraph. 
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We write Gi 0 G 2 to mean the XOR of digraphs Gi and G 2 , (which is to say 
e G Gi 0 G 2 if and only if e G Gi and e ^ G 2 or vice versa.) 

Theorem 2. Let e be any positive eonstant. For any n-node digraph D with 
maximum out-degree A, let D = D(BR where R ~ D„ Then whp Algorithm 
A is eorrect on D when Ni = ^ log(iozi) logn . 

Here Ai and Bi are absolute constants. We find that Ai = 400 and A 2 = 200 
suffice, but we do not attempt to optimize these values. 

If a strongly connected digraph has bounded out-degree and has diameter 
G(log n) then a random walk of length G(log n) has a 1/ poly(n) chance of going 
from s to t, and Algorithm A will succeed whp using values of Ni and N 2 that 
can be realized in log-space. Unfortunately, even though our initial instances have 
bounded out-degree and (as indicated by Theorem 1) our perturbed instances 
have logarithmic diameter, the perturbation increases the maximum degree to 
l7(logn/ log logn), so we must work a little harder to show that the random 
walk has a non-negligible probability of witnessing the path from s to t. (As an 
additional reward for this work, we find that Algorithm A can be derandomized 
by checking all paths from s of length 0{e~^ logn) and still only use log-space.) 

The analysis of Algorithm A is further complicated by the possibility a D 
which is not strongly connected combining with R to produce a smoothed in- 
stance which is strongly connected. We handle this situation by arguing that for 
the smoothed instance to become strongly connected there cannot be too many 
small strong components of D, and then the large components become strongly 
connected with short paths that a random walk is not too unlikely to travel, and 
finally, the small components, if they are connected to the large component, are 
“close” to it. 



Why Study Instances with Bounded Out-Degree? It would be nice to 
extend our results to hold for perturbed copies of any digraph, instead of only 
digraphs with bounded out-degree. However, such a result is not possible for our 
heuristic. We demonstrate the necessity of our assumption that D has bounded 
out-degree by constructing a family of instances with growing degree on which 
Algorithm A does not succeed whp. 

Theorem 3. Let f{n) 00 , let e he a positive eonstant less than 1, and let 
R ~ V'n,e/n- Then for every sufficiently large n, there exists an n-node digraph 
D with maximum out-degree A = f{n) such that for D ® R the probability 
Algorithm A fails exceeds 1 — e~'^ — o(l). 

The proof of Theorem 3 is omitted for lack of space. 



Strong Connectivity Versus (s, t)-Connectivity. Strong connectivity is an 
NL-complete problem, but (s, t)-connectivity is “the” NL-complete problem. In 
Sipser’s undergraduate text [26], the completeness of (s, t)-connectivity is proved 
in detail, while the completeness of strong connectivity is left as an exercise. 
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In light of this, it is natural to investigate the efficacy of heuristics on 
smoothed instances of (s, t)-connectivity. Here we find that there are instances 
on which Algorithm A fails whp. What is more, no log-space heuristic exists, 
provided a conjecture of complexity theory holds. 

Theorem 4. If NL % almost-h then no log-space heuristic succeeds whp on 
smoothed instances of bounded out-degree {s,f}- connectivity. 

The proof consists of building a machine which simulates any nondeterministic 
log-space machine using the log-space heuristic for (s, t)-connectivity, were such 
a heuristic to exist. The proof is omitted for lack of space. 

Smoothed Model Versus Semi-random Model. The semi-random model 
was introduced by Santha and Vazirani in [25] . In this model an adversary adap- 
tively chooses a sequence of bits and each is corrupted independently with prob- 
ability 6. They present this as a model for real-world random bits, such as the 
output of a Geiger counter or noisy diode, and consider the possibility of using 
such random bits in computation on worst-case instances. Blum and Spencer 
considered the performance of a graph coloring heuristic on random and semi- 
random instances in [6] . Subsequent work has uncovered an interesting difference 
between the random and semi-random instances in graph coloring. The work 
of Alon and Kahale [3] developed a heuristic which succeeds whp on random 
instances with constant expected degree, while work by Feige and Kilian [16] 
showed no heuristic can succeed on semi-random instances with expected degree 
(1 — e)logn (they also developed a heuristic for semi-random instances with 
expected degree (1 -I- e) logn). 

In the original semi-random model of Santha and Vazirani, an instance is 
formed by an adaptive adversary, who looks at all the bits generated so far, 
asks for a particular value for the next bit, and gets the opposite of what was 
asked for with probability <5. Several modifications are proposed in Blum and 
Spencer [6] and also in Feige and Krauthgamer [15]. However, all these variations 
maintain the adaptive aspect of the adversary’s strategy, which at low density 
allows too much power; if the error probability p = (1 — e)logn/n then there 
will be roughly n'^ isolated vertices in and the adversary will be able to 
encode a polynomial sized instance containing no randomness. Since we wish to 
consider extremely sparse perturbations, where the error probability p = e/n, 
we cannot allow an adversary as powerful as in the semi-random model. The 
XOR perturbation considered in this paper is equivalent to a natural weakening 
of the semi-random model: making the adversary oblivious. 

1.2 Application: Property Testing 

Property testing provides an alternative weakening of worst-case decision prob- 
lems. It was formalized by Goldreich, Goldwasser, and Ron in [18]. The goal 
in property testing is to design an algorithm which decides whether a instance 
has a property or differs significantly from all instances which have that prop- 
erty. For example, a property tester for strong connectivity in bounded degree 
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digraphs should accept all strongly connected instances and reject all instances 
that are en edges away from being strongly connected. Note that Algorithm A 
(which is designed to work on smoothed random instances) can be converted 
into a property tester: given an instance D and a gap parameter e, we can ran- 
domly perturb D ourselves by adding random edges and then run Algorithm 
A on the perturbed version. This does not yield anything impressive for testing 
strong connectivity, since the undirected connectivity testing results of Goldreich 
and Ron in [19] can be applied to the directed case to produce a constant time 
tester. However, our perturbation approach also yields a property tester for a 
more difficult connectivity problem, fc-linked. 

A digraph is called Minked if for every choice of 2k distinct vertices si, . . . , Sfe, 
t\, . . . ,tk, the graph contains k vertex disjoint paths joining si to ti, Sk to tk- 
Recognizing /c-linked digraphs is NP-complete for fc > 2. In the bounded degree 
property testing version of fc-linked, we are given a constant e and a digraph D 
with maximum out-degree A and our goal is to accept if D is fc-linked and reject 
if D is more than en arcs away from being fc-linked, and we can do whatever if 
D is not /c-linked, but is close to being so. 



A Heuristic for Testing fe-Linkedness. Given D and e, we perturb D by 
flipping a coin for each node, and with probability e/2 adding a out-arc leading 
to a node chosen uniformly at random. Gall this perturbed instance D. Then, 
for each choice of 2k distinct vertices, repeat the following procedure N\ times: 
For i = 1, . . . , A: starting at Si take steps in a random walk on the graph. If 
all k random walks ever reach the correct k terminals via vertex disjoint paths, 
we continue to the next choice of 2k vertices. Otherwise reject. 

Here k is assumed to be fixed, independent of the input. 

Theorem 5. Let e be any positive constant. For any k-linked n-node graph D 
with maximum degree A, the algorithm above accepts whp. 

A few comments regarding the difference between this theorem and Theorems 
1 and 2. Since we perturbed D by adding a 1-out, the maximum out-degree 
of D is bounded by Z\ -|- 1, so we are not faced with one of the additional 
challenges in Theorem 2 (which is addressed by Lemma 1); if k vertex-disjoint 
logarithmic length paths appear then we can immediately conclude that whp 
they will be discovered after a polynomial number of samples. What is more, 
the fact that such paths exist whp follows from a calculation analogous to the 
proof of Theorem 1, that explores disjoint neighborhoods around all 2k terminals 
simultaneously. Also, the analog of the most difficult part of Theorem 2, showing 
that Algorithm A is correct in the case where a disconnected D leads to a strongly 
connected H, is no longer necessary. In the property testing setting, we are not 
required to correctly recognize instances that lead to this situation. It seems as 
though it might be possible to carry out this most difficult part and obtain a 
heuristic for fc-linked that works on smoothed instances, but the details remain 
elusive. 
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2 Proof of Theorem 1 



We now show that if D is strongly connected then the diameter oi D = D + R 
is O(logn) whp. 

Instead of adding a random digraph R with probability p = e/n, we will 
add two random digraphs R' and R!' with arcs appearing independently with 
probability p' and p" respectively. We will take p' = = (1 ~ o{l))e/n and 

p" = The probability of an edge appearing in the union of R' and R" 

is p' + p" — p'p" = ejn. So adding R' and R" to D yields a digraph identically 
distributed with D. 

We will show that whp D contains short paths of a special form, alternating 
between an appropriate number of edges from D and a random edge of R' . This 
is similar to the approach of Bollobas and Chung [10]. 

Let So = To = 0, and let Sq (resp. Tg) be the set of vertices Sg (resp. tg) for 
which there is a path of length at most 32e“^ logn + 1 from s to Sg (resp. from 
tg to t) in D. 

Now, for i > 1, we identify a set Si of neighbors of and then a set S[ of 
neighbors of Si, and do the same for R and T/. 

Let Si (resp. R) be the set of s" ^ U}=o C S'^ (resp. t” ^ Uj=o Tj U Tj) for 
which the arc (s',s") (resp. {t" ,t')) is in R' for some s' G 

Let S'^ (resp. T') be the set of vertices s' (resp. t') for which there is a path 
in D of length at most 

d = 5e“^ + 1 

from some Si G Si to s' (resp. from t' to some ti G R). Let 

I = [log 2 n] . 



I t \ 

Let Zs =\ {Sj U S'')! and Zt 
S=o ' 



U(T,ui;')i. 

i=0 I 



We will now show that 



Pr[^S S or Zt < = o{n ^). 



( 1 ) 



We will argue this assertion only for Zg. The part regarding Zt is proved anal- 
ogously. 

i 

Let C/i = U (Si U S'). For i < i, suppose that \Ui\ < . Let s" be some 

j=o 

vertex not in Ui. Then the probability that there is an arc in R' connecting S'_j^ 
to s" is 1 — (1 — p')l‘^i-il = (1 — o(l))e 

We use the following Chernoff bounds from [20, Theorem 2.1] on the Binomial 
random variable B(n,p) 



Pr(B(n,p) > np + t) < exp 
Pr(B(n,p) <np — t) < exp 



2(np + 1/3) 
2np j 



(2) 

(3) 
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Since the events {s" G Si} are all independent, we let Z = |S'i| and see that 
Z is distributed as B(n — o{n), (1 — o(l))e|S''_^ |/n). So from (3) we have 



Pr 









< e-dS'_il/9^ 



Now, for \Si\ < we will show that IS”'! is likely to be at least 4e“^|S'i|. 
Note that given its size s, the set Si is a random s-subset of the vertices not in 
Ui- Since \Ui\ < there are n — o{n) vertices to choose from. And since the 
out-degree of D is bounded by A, there are at most vertices within distance 
2d of a given vertex. We will call a randomly selected vertex a failure if it is 
at distance less than 2d from a previously selected vertex. Then the probability 
of failure of a particular vertex, conditioned on the success or failure of any 
other vertices is at most (1 -I- o(l))|S'i|A^'*/n. So the number of failures Nf is 
stochastically dominated by B q), where q = (l-|-o(l))|S'i|A^'^n“^. Since we 
are interested in a wide range of values of |S'i|, we bound the same probability for 
large and small values of l^il. If [Ail > we use (2) to obtain the following: 



Pr 



Nf > \S,\q + n^N [ \U,\<n^N^\s, 

I 

< exp / — 



,2/3 



2(|5,|g + nV3/3) 



, ni/3 5 



which relies on the fact that \Si\ < r?N and hence < (1 -b o(l))A^‘^n^/^. 
If I'S'il < then we use the union bound to show 



Pr 



iV/ > 18 ! \s,\ 



< 



is.i\ ^ ^ , 

18 / V n / 



In either case, we have that the failure set has size od^il) with probability 
1 — 0{n~^). We now obtain a lower bound on the size of S[ by ignoring all the 
failure vertices. Note that the neighborhoods of size d of the non-failure vertices 
are disjoint, and since D is strongly connected, there are at least d — 1 vertices 
within distance d of every vertex. So 

\S[\>{l-o{l))be-^\S,\ 

with probability 1 — 0{n~^). Combining this with our bound on the probability 
that \Si\ < 5 e|S''_y gives 



Pr 
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Again using the fact that D is strongly connected, we have |S'ol > 32e ^ logn, 
so we have 



Pr 



\Ui\< 



< Pr 



\Ui\<n^/^ &Yid\S[\<2\S[_^\ 



< ^Pr 



2=1 



l^'l < ! |C/,| < n^/3, |A'_i| > 32e-ilogn 



= 0{n '^logn). 



From the above we have that U = lJi=o ^ = Ui=o have 

size at least with probability 1 — 0{n~^). If U, W intersect, then we know 
there is a path of length at most 2{d + 1)£. Otherwise, we expose the edges of 
R" (which appear independently with probability p" = and find that 

Pr[A" n [/ X IF = 0] < (1 -p")”"''" < e-”"''' = o{n~'^). 



Since there are only n(n — 1) choices for s,t, the theorem follows. □ 



3 Sketch of Proof of Theorem 2 

3.1 When D Is Strongly Connected 

By Theorem 1 we know that the diameter of D is O(logn) whp. Unfortunately 
we cannot yet conclude that Algorithm A is successful whp. We must still 
argue that the probability of a random walk traversing the short path is not too 
small. In the original graph, D, having a diameter of O(logn) would imply an 
efficient algorithm, because the maximum out-degree is bounded. Our random 
perturbation has created some vertices with out-degree f?(log n/ log log n), so we 
will have to work a little more. 

Lemma 1. Let D = D + R, where D is an arbitrary digraph with maximum 
out-degree A and R ~ D„ Then whp D contains no path P of length i < 
£q = 100e“^logn with Oog(iozi)^ 

The proof of Lemma 1 is omitted due to lack of space. 

The correctness of Algorithm A in the case when D is strongly connected 
now follows from that fact that the probability a random walk follows a path P 
from s to t is precisely (IIxgp 

3.2 When D Not Strongly Connected 

The previous section shows that Algorithm A is correct whp for strongly con- 
nected digraphs D. To prove that Algorithm A is correct whp when D is not 
strongly connected, we must do some more work. 

Outline of approach 

Consider the strong components of D. If there are many components of size 
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less than j€~^\ogn, then we show that whp one of them will be incident to 
no arcs of R and so D will not be strongly connected and Algorithm A will 
be correct. In the case where D consists mostly of larger strong components, 
we expose the random arcs R in two rounds. We argue that whp the strong 
components of D merge into a unique giant strong component containing at 
least n — vertices after the first round. Then we invoke Lemma 1 from 

the previous section to show that the random arcs from the second round gives 
the giant a low diameter. Then we deal with the vertices that belong to small 
strong components after the first round of random arcs have been added. These 
vertices might be connected to the giant in both directions and they might not, 
and there is not necessarily a sharp threshold for strong connectivity. However, 
we show that for some constants A\ whp no such vertex is connected in either 
direction to the giant only by paths of length more than A\e~^ logn i.e. such a 
vertex is close to the giant in some direction, or cannot be reached at all in this 
direction. Finally, by Lemma 1 we know that all the paths of length at most 
Aie~^ logn have a non-negligible probability of being traversed by Algorithm A 
(take the bound in Lemma 1 and raise it to the power Ai/lOO). So we conclude 
that whp the graph is not strongly connected, in which case Algorithm A is 
correct, or the graph is strongly connected in such a way that Algorithm A is 
still correct. 

The calculations required for this plan are omitted due to lack of space. 

4 Proof of Theorem 5 

We will show that if D is fc-linked then whp D contains disjoint paths of length 
at most 100fce“^logn which witness its fc-linkedness. 

For a particular choice of si, . . . , Sfc, ti, . . . , tfe, let Qi, Q 2 , ■ • ■ , Qk be vertex 
disjoint paths in D such that Qr goes from Sr to R- 

We order the paths from longest to shortest and define I so that for r < i 
each Qr has length at least 100fce“^ log n. If ^ = 0 then there is nothing to prove, 
so suppose f > I. 

We use the same type of argument as in the proof of Theorem I to show 
the existence of short paths between and Q, but we work will all r < ^ 
simultaneously to ensure that the paths we find are vertex disjoint. To this end, 
we define a sequence of collections of sets Si^r and for t > 0 and I < r < £, 
(and define Si^r = Qi for i > 0 and r > £). Let S'o,r be the first 32ke~^logn 
vertices of Qr and To^r be the last 32ke~^logn vertices of Qr, for 1 < r < £. 

We will call a node useful if it is not within d = 5e~^ of any node which we 
have previously placed in any S or T set. 

To define Si^r, we check, for each node s' in Si-i^r, if s' has a random out-arc 
between it and a useful node s" . If it does, we add s" and all nodes reachable 
from s" in d steps to Si^r- We continue this until exceeds 2|S'i_i,r| (and 
declare failure if it does not) . 

Ti^r is defined analogously, but the paths lead towards tr instead of away. For 
a node t' in we look for useful nodes t" where the random arc is directed 

from t" to t' , and put 2\Ti-i^r\ of them in Ti^r- 
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We wish to argue that we can form sets Si^r and Ti^r of the appropriate sizes 
while = \Ti,r\ < Since every time we discover a useful neighbor we 

add d = 5e“^ nodes, we only see failure if for some i and r, we find that Si^r has 
less than |e|5'i^r| useful neighbors. But at any step, there are at most 
nodes which have been placed in any S or T, and (since the degree is bounded 
by A) this means there must be at least n — useful nodes. So each node 

in S'j j. has a random out-arc to useful neighbor independently with probability 
at least (1 — o(l))e, and using the Chernoff bound (3), we have 



Pr 



The number of useful neighbors of Si^r < 






Also, each useful node has a random out-arc to a node in Ti^r independently 
with probability so by (3) we have 



Pr 



The number of useful neighbors of Ti^r < 2 






Since |S'o,r| = |2o,r| = 32fce“^ logn, a union bound shows that the probability 
of failure is less than 2kn~^^ log n. 

To conclude, we note that when |S'i,r.| = |Tyr| > , the probability that 

no node of has a random out-arc to a node of is at most 



< exp|-^n^/3| = o(n 

So the probability that there are not vertex disjoint paths of length less than 
100e“^fclogn from Si to ti is at most Thus by the union bound, we see 

that all choices of 2k nodes have short edge disjoint paths linking them whp. 
Since the random 1-out perturbation preserves the boundedness of out-degree, 
random walks will discover these paths with non-negligible probability. □ 
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Abstract. Let G(n, c/n) and Griji) be an n-node sparse random and a 
sparse random r-regular graph, respectively, and let X(n,c) and X(n,r) 
be the sizes of the largest independent set in G(n,c/n) and Gr(n). The 
asymptotic value of T(n,c)/n as n oo, can be computed using the 
Karp-Sipser algorithm when c < e. For random cubic graphs, r = 3, it is 
only known that .432 < Ummi„X{n,3)/n < limsup„iT(n, 3)/n < .4591 
with high probability (w.h.p.) as n ^ oo, as shown in [FS94] and [Bol81], 
respectively. 

In this paper we assume in addition that the nodes of the graph are 
equipped with non-negative weights, independently generated accord- 
ing to some common distribution, and we consider instead the maxi- 
mum weight of an independent set. Surprisingly, we discover that for 
certain weight distributions, the limit lim„ T(n, c)/n can be computed 
exactly even when c > e, and lim„ 2i(n, r)/n can be computed exactly 
for some r > 2. For example, when the weights are exponentially dis- 
tributed with parameter 1, lim„iT(n, 2e)/n « .5517 in G(n, c/n), and 
lim„ T(n, 3)/n « .6077 in Gi{n). Our results are established using the 
recently developed local weak convergence method further reduced to a 
certain local optimality property exhibited by the models we consider. 

We extend our results to maximum weight matchings in G(n, c/n) and 
Gr{n). 

1 Introduction 

Two models of random graphs considered in this paper are a sparse random 
graph G(n, c/n) and a sparse random regular graph Gr(n). The first is a graph on 
n nodes {0, 1, . . . , n—1} = [n], where each undirected edge (f, j), 0 < t < j < n— 1 
is present in the graph with probability c/n, independently for all n(n — l)/2 
edges. Here c > 0 is a fixed constant, independent of n. A random r-regular 
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graph Gr(n) is obtained by fixing a constant integer r > 2 and considering a 
graph selected uniformly at random from the space of all r-regular graphs on n 
nodes (graphs in which every node has degree r) . A set of nodes y in a graph G is 
defined to be an independent set if no two nodes of V are connected by an edge. 
Let T(n,c) and T{n,r) denote the maximum cardinality of an independent set 
in G{n,c/n) and Gr{n) respectively. Suppose the nodes of a graph are equipped 
with some non-negative weights Wi,0 < i < n — 1 which are generated indepen- 
dently according to some common distribution Fuj{t) = V{Wi < t),t > 0. Let 
2w(n, c),2w{ti, r) denote the maximum weight of an independent set in G{n, c/n) 
and Gr{n) respectively. 

Let Af(n, c) and A4(n,r) denote the maximum cardinality of a matching 
in G{n,c/n) and Gr(n), respectively. It is known that Gr{n),r > 3 has a full 
matching w.h.p., that is M{n, r) = n/2 ([n/2j for odd n) w.h.p. [JLROO]. If the 
edges of the graph are equipped with some non-negative random weights, then 
we consider instead the maximum weight of a matching c) and Aiw{n, r) 

in graphs G(ji,c/n),Gr{n), respectively. The computation of Xu,(n,c), 
Mw{n,c), A4w(n,r) in the limit as n ^ oo is the main subject of the present 
paper. 

The asymptotic values of M.(n, c) for all c were obtained by Karp and Sipser 
using a simple greedy type algorithm in [KS81]. The result extends to J{n,c) 
but only for c < e. It is an open problem to compute the corresponding limit for 
independent sets for the case c > e or even to show that such a limit exists [Aid] . 
Likewise it is an open problem to compute the corresponding limit in random 
regular graphs. 

The developments in this paper show that, surprisingly, proving the exis- 
tence and computation of the limits lim„X(n, -)/n, lim„ AI(-)/n is easier in the 
weighted case than in the unweighted case, at least for certain weight distribu- 
tions. In particular, we compute the limits for independent sets in Gr{n),r = 
2,3,4 and G{n,c/n),c < 2e, when the node weights are exponentially dis- 
tributed, and we compute the limits for matchings in Gr{n) and G{n,cfn) for 
all r, c, when the edge weights are exponentially distributed. It was shown by the 
first author [Gam04] that the limit lim„ AI(n, -)/n exists for every weight distri- 
bution with bounded support, though the non-constructive methods employed 
prevented the computation of the limits. 

Our method of proof is based on a powerful local weak convergence method 
developed by Aldous [Ald92], [AldOl], Aldous and Steele [AS03], Steele [Ste02], 
further empowered by a certain local optimality observation derived in this pa- 
per. Local weak convergence is a recursion technique based on fixed points of 
distributional equations, which allows one to compute limits of some random 
combinatorial structures, see Aldous and Bandyopadhyay [AB] and Aldous and 
Steele [AS03] for recent surveys on these topics. In particular, the method is used 
to compute maximum weight matching on a random tree, when the weights are 
exponentially distributed. The tree structure was essential in [AS03] for cer- 
tain computations and the approach does not extend directly to graphs like 
G(n, c/n) with c > 1, where the convenience of a tree structure is lost due to 
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the presence of a giant component. It was conjectured in [AS03] that a some 
long-range independence property might be helpful to deal with this difficulty. 
The present paper partially answers this qualitative conjecture in a positive 
way. We introduce a certain operator T acting on the space of distribution func- 
tions. We prove a certain local optimality property stating that, for example, 
for independent sets, whether a given node i belongs to the maximum weight 
independent set is asymptotically independent from the portion of the graph 
outside a constant size neighborhood of i, iff has a unique fixed point dis- 
tribution. Moreover, when does have the unique fixed point, the size of the 
extremal object (say maximum weight independent set) can be derived from a 
fixed point of an operator T. The computations of fixed points is tedious, but 
simple in principle and the groundwork for that was already done in [AS03]. 
We hope that the long-range independence holds in other random combinatorial 
structures as well. In fact the issues of long-range independence were already 
considered by Aldous and Bandyopadhyay [AB] , Bandyopadhyay [Ban03] using 
the notion of endogeny. This specific version of long-range independence turned 
out to be critical in Aldous [AldOl] for proving the C(2) limit of the random 
assignment problem. 

The issue of long-range independence (usually called correlation decay in- 
stead) of random combinatorial objects is addressed in a somewhat different, 
statistical physics context in Mossel [Mos03], Brightwell and Winkler [BW03], 
Rozikov and Suhov [RS03], Martin [Mar03], Martinelli, Sinclair and Weitz 
[MSW03], where independent sets (hard-core model) and other combinatorial 
models are considered on regular trees, weighted by the Gibbs measure. In a dif- 
ferent setting Talagrand [Tal03] proves a certain long-range independence prop- 
erty for the random assignment problem, where the usual min-weight matching 
is replaced by a partition function on the space of feasible matchings. He uses a 
rigorous mathematical version of the cavity method, which originated in physics, 
to prove that the spins are asymptotically independent as the size of the problem 
increases. The particular form of the long-range independence is similar to the 
one we obtain, and in fact the cavity method has some similarity with the local 
weak convergence method, which is also based on considering extremal objects 
(say independent sets) with one or several nodes excluded. Finally, we refer the 
reader to Hartmann and Weigt [HWOl] who derive the same result as Karp 
and Sipser for independent sets using non-rigorous arguments from statistical 
physics. 

We finish this section with some notational conventions. Exp(/i), Pois(A) de- 
note respectively exponential and Poisson distributions with parameters A > 
0, 0 < z < 1. Most of the proofs are too lengthy to fit under the page limitation 
and we refer the reader to [GNS03] for proofs. 

2 Prior Work and Open Questions 

It is known and simple to prove that the largest independent sets are 0{n) 
w.h.p. in both G{n,cfn) and Gr{n) when c and r are any constants inde- 
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pendent from n. Moreover, it is known that in random cubic graphs, w.h.p., 
61og(3/2) — 2 = .432 . . . < liminf„I(n, 3)/n < limsup„X(n, 3)/n < .4591. The 
lower bound is due to Frieze and Suen [FS94], and the upper bound is due 
to Bollobas [BolSl]. The upper bound is generalized for any r > 1 and uses 
a very ingenious construction of random regular graphs via matching and ran- 
dom grouping, [BolSO], [JLROO]. It is natural to expect that the following is true, 
which unfortunately remains only a conjecture, appearing in several places, most 
recently in [Aid] and [AS03]. 

Conjecture 1 For every c > 0 and r > 3 the limits 

to to 

n— ^oo 77, n— ^oo 77, 



exist. 

The existence of these limits also implies the convergence to the same limits 
w.h.p. by applying Azuma’s inequality, see [JLROO] for the statement and the 
applicability of this inequality. 

The limits lim„ EX(n, c) /n, lim„ EA4(n, c)/n are known to exist for for inde- 
pendent sets when c < e and for matching for all c and were first derived by 
Karp-Sipser [KS81]. Their results were strengthened later by by Aronson, Frieze 
and Pittel [APF98]. 

The algorithm underlying Karp and Sipser analysis unfortunately is not ap- 
plicable to random regular graphs. Moreover, if the edges or the nodes of the 
graph G{n, c/n) are equipped with weights then the Karp-Sipser algorithm can 
produce a strictly suboptimal solution and cannot be used in our setting of 
weighted nodes and edges. Also, when the edges of Gr{n) are equipped with 
weights, the problem of computing maximum weight matching becomes non- 
trivial, as opposed to the unweighted case when the full matching exists w.h.p. 

In a somewhat different domain of extremal combinatorics the following result 
was established by Hopkins and Staton [HS82] . A girth of a graph is the size of 
the smallest cycle. It is shown in [HS82] that the size of a largest independent 
set in an n-node graph with largest degree 3 and large girth is asymptotically 
at least (7/18)n — o{n). The techniques we employ in this paper allow us to 
improved this lower bound. 



3 Main Results 

We begin by introducing the key technique for our analysis - recursive distri- 
butional equations and fixed point solutions. This technique was introduced by 
Aldous [Ald92], [AldOl] and was further developed in Aldous and Steele [AS03], 
Steele [Ste02], Aldous and Bandyopadhyay [AB]. Let IF be a non-negative ran- 
dom variable with a distribution function F^it) = P(IF < t). We consider four 
operators T = Tx,r> Tx,c, Tm,v, Tm,c acting on the space of distribution functions 
F{t),t > 0, where c > 0 is a fixed constant and r > 2 is a fixed integer. 
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1. Given W distributed according to (we write simply W ~ F^), and given 
a distribution function F = F(t), let Bi, B 2 , ■■■, Br ^ F be generated inde- 
pendently. Then Tx,r : F ^ F' , where F' is the distribution function of B' 
defined by 

B' = max(0, W — E B.). (1) 

l<z<r 

2 . Under the same setting as above, let Bi, , Bm ~ F, where m is a random 
variable distributed according to a Poisson distribution with parameter c, 
independently from W, Bi. Then Tx,c : F F', where F' is the distribution 
function of B' defined by 

B' = max(0, W — E (2) 

K'i<m 



when m > 1 and B' = W when m = 0. For simplicity we identify the sum 
above with zero when m = 0. 

3. Let Wi, . . . , IVr ~ Fw, Bi, . . . ,Br ~ F. Then Tm.t '■ F F', where F' is 
the distribution function of B' defined by 

eq : RecursionMrB' = max (0, Wi — Bi). (3) 

l<z<r 

from Wi,Bi. Then Tx,c : F ^ F' , where F' is the distribution function of 
B' defined by 

B' = ma.x {0,Wi- B,), (4) 

l<z<m 

when TO > 1 and B' = 0 when to = 0. Again, for simplicity, we assume that 
the max expression above is zero when to = 0. 

A distribution function F is defined to be a fixed point distribution of an 
operator T if T{F) = F. 

We now state the main result of this paper. Recall, that a distribution func- 
tion F(t) is defined to be continuous (atom free) if for every x in its sup- 
port lim£^o(.F(a^ + c) ~ ~ c)) = 0- Equivalently, for B ^ F and every 

X, ¥{B = x) = 0. We use !{•} to denote the indicator function. 

Theorem 1. Let F^j he a continuous non-negative distribution function. For 
r > 1 if the operator has a unique fixed point distribution function F* , 

then, w.h.p. 

=E[W1{W- V (5) 

l<z<r 

where W ~ F^,, Bi ^ F* , and W, Bi are independent. When Gr{n) is replaced by 
G{n,cjn), the same result holds for T = Tj,c, except the sum in the right-hand 
side of (5) is Bi and m ~ Pois(c). 
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Finally, the similar results hold for A4w(n,r) and A4w(n,c) in Gr{n) and 
G(n,c), for T = and T = Tm,c, respectively, whenever the correspond- 

ing operator T is such that has the unique fixed point distribution F* . The 
corresponding limits are 

lim ^ V W, 1{W, -B,= max {Wj - B,) > 0}], (6) 

n 71 Z Kj<r 

l<i<r 

where Wi ~ Bi ^ , and 

lim = 1e[ V W, l{Wi -B,= max{W, - Bj) > 0}], (7) 

n n z 

i<.m 



where Wi ^ F,„,Bi ^ F* ,m ^ Pois(c). 



The Theorem 1 is the core result of this paper. It will allow us to obtain several 
interesting corollaries, which we state below. The rest of the paper gives a coarse 
outline of the proof of Theorem 1. A detailed proof is found in [GNS03]. 

Theorem 2. Suppose the weights of the nodes and edges of the graphs G = 
Gr{n) and G = G{n,c) are distributed as Exp(l). Then 

a unique fixed point distribution F* iff r < 4. In this case, w.h.p. 
Iw{n,r) (1 — 6)(r — r& + 2& + 2) 



lim ■ 



(8) 



where b is the unique solution of b = 1 — (-^)^ In particular^ w.h.p. 



lim 



Iw{n,2) _ 2 
n 3’ 



hm ^"(^’^) ^.6077, 



lini « .4974, (9) 



n 



2. Tf ^ has a unique fixed point distribution F* iff c < 2e. In this case, w.h.p. 






(10) 



where b is the unique solution ofl — b= In particular, when c = 2e, 

this limit is « .5517. 

3. T^ has a unique fixed point F* for every r >2. Moreover, w.h.p. 



lim 



Mw(n,r) 



pOC) 

= r{V~^ + 1) / te~\l - e~\l - b))y-^dt (11) 

Jo 

pOC) 

-r te-\l-e-\l-b))f^-‘^dt, 

Jo 



where b is the unique solution of b = 1 



l-b’’ 

TWF)- 
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4- ^ has a unique fixed point F* for all c > 0. Moreover, for every c > 0 

w.h.p. 



lim 



M^„{n,c) c 



= £(e^'>-" + l) f "dt 

2 Jo 



(12) 



te 



-t-2c(l-6)e“*+c(l-6)^ 



dt. 



where b is the unique solution of 1 — e “ = c(l — 6)^. 



The proof of Theorem 2, specifically the corresponding computations of the 
limits is quite lengthy and can be found in [GNS03]. The expression in (11) 
involving integrals is similar to the one found in [AS03] for maximum weight 
matching on a tree. It is a pleasant surprise, though, that the answers for inde- 
pendent sets are derived in closed form. 

Part 2 of Theorem 2 leads to an interesting phase transition behavior. For 
ifu, = Exp(l) our result says that the value of c = 2e is a phase transition 
point for the operator Tj ^ where for c < 2e the operator has a unique fixed 
point distribution, but for c > 2e the fixed point distribution is not unique. It 
turns out (see Theorems 4, 5 below) that this phase transition is directly related 
to some long range independence/dependence property of the maximum weight 
independent sets in the underlying graph G(n, c/n). Interestingly, no such phase 
transition occurs for maximum weight matchings. 

The methods developed in this paper allow us the improve the following 
result of Hopkins and Staton [HS82]. Let Q{n,r,d) denote the class of all (non- 
random) graphs on n nodes, with maximum degree r and girth at least d. For 
any G G Q{n,r,d) let X(G) denote the size of the largest independent set in G. 
Hopkins and Staton proved that lim inf„^d minGgp(„ 3 j;) T(G)/n > 7/18 « .3887. 
Our techniques allow us to obtain the following improvement. 



Theorem 3. 

lim inf min > .3923. 

n,d GGQ{n,3,d) Tl 



For the proof of this theorem see [GNS03]. 



4 Fixed Points of the Operator 

and the Long-Range Independence 

4.1 Maximum Weight Independent Sets and Matchings in Trees. 
Fixed Points of and the Bonus Function 

We start by analyzing operator T ~ any of the four operators introduced in the 
previous section. Given two distribution functions F\,F 2 defined on [0,oo), we 
say that F 2 stochastically dominates F\ and write Fx ^ F 2 if Fi(t) > F 2 {t) 
for every t > 0. A sequence of distribution functions is defined to converge 
weakly to a distribution function F (written F) if lim„F„(t) = F{f) for 
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every t which is a point of continuity of F. All the convergence of distributions 
is understood here in a weak sense. 

Let 0 denote (for simplicity) the distribution function of a random variable X 
which is zero w.p.l. Let = maxi<i<r Wi where Wi ~ are independent. 

Let also = maxi<i<m Wi, where Wi ~ F^ are independent and m ~ 

Pois(c). Denote by and Fyj^c. the distribution functions of and 

respectively. 

Proposition 1. Fix r > l,c > 0, a distribution function Fyj{t),t > 0 and 
T = Tj^r or Tj^c- As s ^ oo, the two sequences of distributions T^®(0), 
weakly converge to some distribution functions f** , F** , respectively, which are 
fixed points of the operator T^. For any distribution function Fq = Fo(t),t > 0, 
y 2 s(o) ^ t2*+2(Fo) ^ T2.(F„) and T2'*+1(F„) ^ T^^+^{Fo) -< T‘^'^+^0) for all 
s=l,2,.... 

If the operator T is such that has a unique fixed point ( and F** = F** = 
F* ), then for any distribution function Fq = Fo(t), T^(Fo), s = 1,2,... converges 
to F* as s ^ oo. In particular, T^(0), T®(Fu,) ^ F* . Moreover, F* is also the 
unique fixed point ofT. When T = Tm,t orT = the same result holds with 
Fyj^r and F^^c respectively replacing Fyj. 

Proof. See [GNS03]. 

We now switch to analyzing the maximum weight independent set problem 
on a tree. The derivation here repeats the development in [AS03] for maximum 
weight matching in random trees. We highlight important differences where ap- 
propriate. 

Suppose we have a (non-random) finite tree H with nodes 0,l,...,/i = |iL| — 
1, with a fixed root 0. The nodes of this tree are equipped with some (non- 
random) weights Wo, Wi, . . . , Wh > 0. For any node i € H, let H{i) denote the 
subtree rooted at i consisting of all the descendants of i. In particular, H (0) = H. 
Let denote the maximum weight of an independent set in H{i) and let 

Bh(i) = where the sum runs over nodes j which are children 

of i. If i has no children then is simply lH{i) = Wi. Observe, that BH{i) is 

also a difference between ^H(i) and the maximum weight of an independent set 
in H{i), which is not allowed to use node i. Clearly, 0 < < Wi. The value 

Bh({) was considered in [AS03] in the context of maximum weight matchings and 
was referred to as a bonus of a node i in tree H{i). W.l.g. denote by 1, . . . ,m 
the children of the root node 0. 

Lemma 1. 

Bh(o) = max(0, Wq - E (13) 

l<z<m 

Moreover, if Wq > (lhat is if Bu{p) > Oj then the maximum 

weight independent set must contain node 0. If Wq < then the 

maximum weight independent set does not contain the node 0. 



Proof See [GNS03]. 
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A similar development is possible for maximum weight matching. Suppose 
the edges of the tree H are equipped with weights Wij. Let denote the 

maximum weight of a matching in H(i), and let denote the difference 

between M.h(i) the maximum weight of a matching in H{i) which is not 
allowed to include any edge in H{i) incident to i. Again 1, 2, . . . , m are assumed 
to be the children of the root 0. 

Lemma 2. 

Bh = max(0, max (ILoy - Bh{^)). (14) 

l<z<m 

Moreover, if Wo.i - BH(i) > ~ BH(i') for all i' yf i and Woy - BH(i) > 0, 

then every maximum weight matching contains edge (0,f). IfWo^i — BH{i) < 0 
for all i = 1, . . . ,m, then every maximum weight matching does not contain any 
edge incident to 0. 

Proof See [GNS03]. 



4.2 Long-Range Independence 

We now consider trees H of specific types. Given integers r > 2, d > 2 let Hr{d) 
denote an r-regular finite tree with depth d. The root node 0 has degree r — 1, 
all the nodes at distance > 1,< d — 1 from the root have outdegree r — 1, 
and all the nodes at distance d from 0 are leaves. (Usually, in the definition of 
an r-regular tree, the root node is assumed to have degree r, not r — 1. The 
slight distinction here is done for convenience.) Also, given a constant c > 0, 
a Poisson tree id(c, d) with parameter c and depth d is constructed as follows. 
The root node has a degree which is a random variable distributed according 
to Pois(c) distribution. All the children of 0 have outdegrees which are also 
random, distributed according to Pois(c). In particular, the children of 0 have 
total degrees 1 -I- Pois(c). Similarly, children of children of 0 also have outdegree 
Pois(c), etc. We continue this process until either the process stops at some depth 
d' < d, where no nodes in level d' have any children, or until we reach level d. 
In this case all the children of the nodes in level d are deleted and the nodes in 
level d become leaves. We obtain a tree with depth < d. We call this a depth-d 
Poisson tree. 

Let H = Hr{d) or H{c,d). Suppose the nodes and the edges of H are 
equipped with weights Wi,Wij, which are generated at random independently 
using a distribution function Fw Fix any infinite sequences w = {w\,W 2 , . . .) G 
[0, oo)°° and b = ( 6 i, 62 , • ■ .) G {0, 1}°°. For every i = 1, 2, . . . , d let il,i2,. . iji 
denote the nodes of H in level i (if any exist for H{c, d)). When H = Hr{d),ji = 
(r — 1)*, of course. Let (T|(6, w)) denote the maximum weight of an independent 
set V in FI such that the nodes dj with bj = 1 are conditioned to be in V, 
nodes dj with bj = 0 are conditioned not to be in V, and the weights of nodes 
dj are conditioned to be equal to Wj for j = 1, ... ,jd- That is we are looking 
for maximum weight of an independent set among those which contain depth 
d leaves with bj = 1, do not contain depth d leaves with bj = 0, and with the 
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weights of the leaves deterministically set by w. For brevity we call it the maxi- 
mum weight of an independent set with boundary condition (b,w). For the case 
H = H{c,d), the boundary condition is simply absent when the tree does not 
contain any nodes in the last level d. {Juiij) l(^) w)) are defined similarly for the 
subtrees H{ij) spanned by nodes ij in level i. Given again b,w, let (At |(6, tu)) 
and {M.H(ij)\{b,w)) denote, respectively, the maximum weight of a matching E 
in H and Hij, such that the edges incident to nodes dj are conditioned to be in 
E when bj = 1, edges incident to nodes dj are conditioned not to be in D when 
with bj = 0, and the weights of the edges incident to nodes dj are conditioned 
to be equal to Wj,j = 1, . . . ,jd (of course, we refer to edges between nodes in 
levels d — 1 and d as there is only one edge per each node in level d) . 

For the case of independent sets, let (i?|(6, tu)) denote the bonus of the root 
node 0 given the boundary condition {b,w). Namely, 

{B\(b,w)) = {J\(b,w)) - i^H{ij)\(b,w)). 

For the case of matchings, let {B\{b,w)) also denote the bonus of the root node 
0 given the boundary condition (b,w). Namely, 

(B|(6,u;)) = (Af|(6,w)) - ^ {MH{ij)\{b,w)). 

It should be always clear from the context whether B is taken with respect to 
independent sets or matchings. 

The following theorem establishes the crucial long-range independence prop- 
erty for the maximum weight independent sets and matchings in trees El = 
Hr{d), H{c,d) when the corresponding operator has a unique fixed point. It 
establishes that the distribution of the bonus (B\{b,w)) of the root is asymptot- 
ically independent of the boundary condition {b,w) as d becomes large. Recall 
our convention that 0 denotes the distribution of a random variable which is 
equal to zero with probability one. 

Theorem 4. Given a distribution function E^ and a regular tree H = Hr{d). 
Suppose the operator T'^ = Txr-i unique fixed distribution F* . Then 

sup |p((R|(&,u;)) <t)~ F*{t) \ 0, (15) 

b,w 

as d — > oo. Similar assertion holds for T = T = Tx,c and T = For 

the cases T = r and T^^c, Fyj is replaced with and F^^c respectively. 

Let us compare this result with the developments in ([AS03]). In that paper 
maximum weight matching is considered on an n-node tree, drawn independently 
and uniformly from the space of all labelled trees. The notion of a bonus 
is introduced and the recursion (14) is derived. However, since a tree structure 
is assumed to begin with, there is no need to consider the boundary conditions 
(6, w). Here we avoid the difficulty of the non-tree structure by proving the long- 
range independence property via the uniqueness of fixed points of T^. 
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Proof. See [GNS03]. 

While it is not used in this paper, it is interesting that the uniqueness of 
the solution T^{F) = F is the tight condition for (15), as the following theorem 
indicates. 

Theorem 5. Under the setting of Theorem 5 suppose the operator has more 
than one fixed point distributions F* . Then for every such F* 

liminf sup 'P((i3|(&, iZ;)) < t) — F*(t)< > 0. (16) 

d- b,w' ' 



Proof See [GNS03]. 

4.3 Applications to Maximum Weight Independent Sets 
and Matchings in Gr{n) and G{n, c/n) 

The goal of the current section is to demonstrate that Theorem 4 allows us 
to reduce the computation of the maximum weight independent set and the 
maximum weight matching in random graphs to a much simpler problem of 
finding those in trees. We highlight this key message of the paper as the following 
local optimality property: if the operator corresponding to a maximum weight 
combinatorial object (independent set or matching) in a sparse random graph has 
a unique fixed point, then for a randomly selected node (edge) of the graph, the 
event ’’the node (edge) belongs to the optimal object” and the distribution of the 
node (edge) weight, conditioned that it does, asymptotically depends only on the 
constant size neighborhood of the node and is independent from the rest of the 
graph. In other words, when T^ has a unique fixed point, the maximum weight 
independent sets and matchings exhibit a long-range independence property. 
Our hope is that similar local optimality can be established for other random 
combinatorial structures. A complete proof of Theorem 1 is given in [GNS03]. 
Here we just give a high level idea. We observe that by symmetry E[Iu)(^^, ?")] = 
nE[Wol{0 € W}], where W is the independent set which achieves the maximum 
weight. We fix a large constant d and consider a size-d neighborhood of the 
node 1. With probability approaching unity this neighborhood is an r-regular 
tree. We then invoke Theorem 5 to argue that the quantity E[lTol{0 G Vj.}] 
is determined almost exactly just by the d-neighborhood of the node 1 and is 
given asymptotically as E[Wol{lTo — X)i<i<r > ^}] where Bi,. . . ,Br are i.i.d 
distributed as F* and F* is the unique fixed point of the operator T| 
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Abstract. The problem of estimating the frequency moment Fk for any non- 
negative k, over a data stream hy looking at the items exactly once as they arrive, 
was considered in a seminal paper hy Alon, Matias and Szegedy [1,2]. The space 
complexity of their algorithm is ). For k > 2, their technique does not 

apply to data streams with arbitrary insertions and deletions. In this paper, we 
present an algorithm for estimating Fk for fe > 2, over general update streams 
whose space complexity is 0(n^~ ) and time complexity of processing each 

stream update is 0(1). 

Recently, an algorithm for estimating Fk over general update streams with sim- 
ilar space complexity has been published by Coppersmith and Kumar [7]. Our 
technique is, (a) basically different from the technique used by [7], (b) is simpler 
and symmetric, and, (c) is more efficient in terms of the time required to process 
a stream update (0(1) compared with 0(n )). 



1 Introduction 

A data stream can be viewed as a sequence of updates, that is, insertions and deletions 
of items. Each update is of the form {I, ±w), where, I is the identity of the item and v 
is the change in frequency of I such that |r;| > 1. The items are assumed to draw their 
identities from the domain [N] = {0,l,...,fV— Ij.Ifwis positive, then the operation 
is an insertion operation, otherwise, the operation is a deletion operation. The frequency 
of an item with identity I, denoted by //, is the sum of the changes in frequencies of 
I from the start of the stream. In this paper, we are interested in computing the 
frequency moment Fk = ft , for k > 2 and k integral, by looking at the items 
exactly once when they arrive. 

The problem of estimating frequency moments over data streams using randomized 
algorithms was first studied in a seminal paper by Alon, Matias and Szegedy [1,2]. 
They present an algorithm, based on sampling, for estimating Fk, for fc > 2, to within 
any specihed approximation factor e and with confidence that is a constant greater than 
1/2. The space complexity of this algorithm is s = (suppressing the term 

and time complexity per update is where, n is the number of distinct 

elements in the stream. This algorithm assumes that frequency updates are restricted to 
the form (/, +1). 

One problem with the sampling algorithm of [1,2] is that it is not applicable to 
streams with arbitrary deletion operations. For some applications, the ability to handle 

K. Jansen et al. (Eds.): APPROX and RANDOM 2004, LNCS 3122, pp. 369-380, 2004. 
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deletions in a stream may be important. For example, a network monitoring applica- 
tion might be continuously maintaining aggregates over the number of currently open 
connections per source or destination. 

In this paper, we present an algorithm for estimating Fk, for k > 2, to within an 
accuracy of (1 ± e) with confidence at least 2/3. (The method can be boosted using the 
median of averages technique to return high conhdence estimates in the standard way 
[1,2].) The algorithm handles arbitrary insertions and legal deletions (i.e., net frequency 
of every item is non-negative) from the stream and generalizes the random linear com- 
binations technique of [1,2] designed specifically for estimating F 2 . The space com- 
plexity of our method is 0{n^~T^) and the time complexity to process each update is 
0(1), where, functions of k and e that do not involve n are treated as constants. 

In [7], Coppersmith and Kumar present an algorithm for estimating Fk over general 
update streams. Their algorithm has similar space complexity (i.e., 0(n^“^^)) as the 
one we design in this paper. The principal differences between our work and the work 
in [7] are as follows. 

1. Different Technique. Our method constructs random linear combinations of the 
frequency vector using randomly chosen roots of unity, that is, we construct the 
sketch Z = fixi, where, xi is a randomly chosen root of unity. Coppersmith 
and Kumar construct random linear combinations C = fiXi, where, for I G [N], 
xi = —l/n ■^^orl — 1/n with probability 1 — 1/n '^^andl/n 
respectively. 

2. Symmetric and Simpler Algorithm. Our technique is a symmetric method for all 

k > 2, and is a direct generalization of the sketch technique of Alon, Matias and 
Szegedy [1,2]. In particular, for every k > 2, E[Re = Fk. The method of 
Coppersmith and Kumar gives complicated expressions for estimating Fk, for k > 
4. For fc = 4, their estimator is (where. Bn « 

and requires, in addition, an estimation of F 2 to within an accuracy factor of (1 ± 
j.j-i/3) jjjg estimator expression for higher values of k (particularly, for powers 
of 2) are not shown in [7]. These expressions require auxiliary moment estimation 
and are quite complicated. 

3. Time Efficient. Our method is significantly more efficient in terms of the time taken 
to process an arrival over the stream. The time complexity to process a stream 
update in our method is 0(1), whereas, the time complexity of the Coppersmith 
Kumar technique is 0(n^“^^). 

The recent and unpublished work in [11] presents an algorithm for estimating Fk, for 
k > 2 and for the append only streaming model (used by [1,2]), with space complexity 
0(n^“'w). Although, the algorithm in [11] improves on the asymptotic space com- 
plexity of the algorithm presented in this paper, it cannot handle deletion operations 
over the stream. Further, the method used by [11] is significantly different from the 
techniques used in this paper, or from the techniques used by Coppersmith and Kumar 

[7]. 

Lower Bounds. The work in [1,2] shows space lower bounds for this problem to be 
D{v}~^/^), for any k > 5. Subsequently, the space lower bounds have been strength- 
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ened to for fc > 2, e > 0, by Bar-Yossef, Jayram, Kumar and Sivaku- 

mar [3], and further to by Chakrabarti, Khot and Sun [5]. Saks and Sun [14] 

show that estimating the Lp distance d between two streaming vectors to within a factor 
of requires space 

Other Related Work. For the special case of computing F 2 , [1, 2] presents an 0(log n+ 
log m) space and time complexity algorithm, where, m is the sum of the frequencies. 
Random linear combinations based on random variables drawn from stable distributions 
were considered by [13] to estimate Fp, for 0 < p < 2. The work presented in [9] 
presents a sketch technique to estimate the difference between two streams based on 
the Li metric norm. There has been substantial work on the problem of estimating Fq 
and related metrics (set expression cardinalities over streams) for the various models of 
data streams [10, 1,4, 12]. 

The rest of the paper is organized as follows. Section 2 describes the method and 
Section 3 presents formal lemmas and their proofs. Finally we conclude in Section 4. 



2 An Overview of the Method 

In this section, we present a simple description of the algorithm and some of its proper- 
ties. The lemmas and theorems stated in this section are proved formally in Section 3. 
Throughout the paper, we treat k as a hxed given value larger than 1 . 



2.1 Sketches Using Random Linear Combinations of Roots of Unity 



Let a: be a randomly chosen root of the equation = 1, such that each of the k roots 
is chosen with equal probability of 1 /k. Given a complex number z, its conjugate is 
denoted by z. For any j, I < J < k, the following basic property holds, as shown 
below. 



E[a;^] =E[x^' 



0 if 1 < j < fc 

1 if j = k. 



( 1 ) 



Proof. Let J = k. Then, E[x-’ ] = E[x^] = E[l] = 1, since, x is a root of unity. 
Let 1 < j < k and let u be the elementary k*^ root of unity, that is, u = 



E[x^' 




1=1 



1 

k 



1=1 



(1 - 

k — u^) 



where, the last equality follows from the sum of a geometric progression in the complex 
held. Since = 1, it follows that = 1. Further, since u is the elementary kd^ root 
of unity, ^ 1 < j < fc. Thus, the expression (1 — — 

u^) = 0. Therefore, E [a;^] = 0, for 1 < j < fc. 

The conjugation operator is a 1 - 1 and onto operator in the held of complex numbers. 
Further, if a; is a root of x^ = 1, then, a;^ = a:^ = 1 = 1, and therefore, x is also a k*^ 
root of unity. Thus, the conjugation operator, applied to the group of roots of unity, 
results in a permutation of the elements in the group (actually, it is an isomorphism). It 
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therefore follows that the sum of the powers of the roots of unity is equal to the sum 
of the powers of the conjugates of the roots of unity. Thus, E [x^] = E [x-’j . □ 

Let Z be the random variable dehned as Z = X)/6 [at] The variable x/, for 
each I G [N], is one of a randomly chosen root of x^ = 1. The family of variables {x/} 
is assumed to be 2fc-wise independent. The following lemma shows that Re is an 
unbiased estimator of Following [1, 2], we call Z as a sketch. The random variable Z 

can be efficiently maintained with respect to stream updates as follows. First, we choose 
a random hash function 6* : [A^] ^ [fc] drawn from a family of hash functions that is 2 k- 
wise independent. Further, we pre-compute the roots of unity into an array Gl[l..fc] 
of size k (of complex numbers), that is, A[r] = for r = 1, 2, . . . , fc. For 

every stream update {I, v), we update the sketch as follows. 

Z=Z + vA[6{l)] 

The space required to maintain the hash function 9 = 0{k), and the time required for 
processing a stream update is also 0{k). 

Lemma 1. E[/?e Z^] = F^. 

As the following lemma shows, the variance of this estimator is quite high. 

Lemma 2. Var[ReZ'=] =0{k^^F^). 

This implies that Var [Re Z^]/(E[Re Z^])^ = 0{F2 / F^), which could be as large 
as To reduce the variance we organize the sketches in a hash table. 

2.2 Organizing Sketches in a Hash Table 

Let (f) : {0,1,..., iV— 1} — > [S] be a hash function that maps the domain (0,1,..., N— 
1} into a hash table consisting of B buckets. The hash function <j) is drawn from a 
family of hash functions H that is 2fc-wise independent. The random bits used by the 
hash family is independent of the random bits used by the family |x;}/g{o,i,...,w-i}> OF 
equivalently, the random bits used to generate (j) and 6 are independent. The indicator 
variable y; for any domain element I G {0, 1, . . . , TV — 1}] and bucket b G [B], is 
dehned as yi^t, = 1 if = b and yi^t, = 0 otherwise. Associated with each bucket b is 
a sketch Zt of the elements that have hashed to that bucket. The random variables, Yf, 
and Zf, are dehned as follows. 

Zb = Y.fi-xi- yi,b, Yb = Re Z^ and E = ^ ^ 

I be[B] 

Maintaining the hash table of sketches in the presence of stream updates is analogous 
to maintaining Z. As discussed previously, let 6 : (0, 1, . . . , — 1} — > [k] denote a 

random hash function that is chosen from a 2fc-wise independent family of hash func- 
tions (and independently of the bits used by (j>), and let A[1 ... A:] be an array whose 
entry is for j = 1, . . . , A:. For every stream update ((, v), we perform the 

following operation. 



^0(0 — + V ■ A[0(T)] 
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The time complexity of the update operation is 0{k). The sketches in the buckets except 
the bucket numbered 4>{l) are left unchanged. 

The main observation of the paper is that the hash partitioning of the sketch Y into 
{Yb}be[B] reduces the variance of Y significantly, while maintaining that E[F] = F^. 
This is stated in the lemma below. 

Lemma 3. Let B < 2v}~^. Then, Var[y] = 0{F^n^~^ / 

A hash table organization of the sketches is normally used to reduce the time com- 
plexity of processing each stream update [6,8]. However, for k > 2, the hash table 
organization of the sketches has the additional effect of reducing the variance. 

Finally, we keep si independent copies H[0], . . . , H[si — 1] of the variable Y. The 
average of these variables is denoted by Y; thus Var[F] = (l/si)Var[y]. The re- 
sult of the paper is summarized below, which states that Y estimates Fk to within an 
accuracy factor of (1 ± e) with constant probability greater than 1/2 (at least 2/3). 

Theorem 1. Letv}~'’^ < B < 2 ■ and si = 6 • 2^ • Then, 

Pv {\Y - Fk\ > eFk} < 1/3. 

The space usage of the algorithm is therefore 0{B ■ si) = 0{v}~~^^) bits, since 
a logarithmic overhead is required to store each sketch Zb. To boost the confidence of 
the answer to at least 1 — a standard technique of returning the median value 

among S 2 such average estimates can be used, as shown in [1,2]. 

The algorithm assumes that the number of buckets in the hash table is B, where, 
n < B < 2 ■ n . Since, in general, the number of distinct items in the 

stream is not known in advance, one possible method that can be used is as follows. 
First estimate n to within a factor of (1 ± 1) using an algorithm for estimating Fg, 
such as [10, 1,2,4]. This can be done with high probability, in space 0(log7V). Keep 
2 log -1-4 group of (independent) hash tables, such that the group uses Bi = [2*^^] 
buckets. Each group of the hash tables uses the data structure described earlier. At the 
time of inference, first n is estimated as n, and, then, we choose a hash table group 
indexed by i such that i = 2 • [(1 — -j^) log (8 • n/7)]. This ensures that the hash 

table size Bi satisfies n < Bi < 2 ■ n , with high probability. Since, the 

number of hash table groups is 2 • log N, this construction adds an overhead in terms 
of both space complexity and update time complexity by a factor of 2 • log N. In the 
remainder of the paper, we assume that n is known exactly, with the understanding that 
this assumption can be alleviated as described. 



3 Analysis 

The frequency moment of the set of elements that map to bucket b under the hash 
function (j), is a random variable denoted by Fj Thus, Fj f, = fiUi.b- Further, since 
every element in the stream hashes to exactly one bucket, Fj f, = Fj. We define hi^b, 
for / G {0, 1, . . . , /V - 1} and b G [B] to be hi^b = fi ■ yi,b- Thus, Fj^b = M for 
J > 1- 
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Notation: Marginal Expectations. The random variables, Y, {Yhlbe b are functions of 
two families of random variables, namely, x = {a;i}jg{o,i,...,Ar-i}> used to generate the 
random roots of unity, and y = I G {0, 1, . . . , — 1} and b G [B], used to map 

elements to buckets in the hash table. Our independence assumptions imply that these 
two families are mutually independent (i.e., their seeds use independent random bits), 
that is, Pr {x = u and y = v} = Pr {x = u} • Pr {y = v} Let W = kL(x, y) be a 
random variable that is a function of the random variables in x and y. For a fixed random 
choice of y = yo, Ex [W] denotes the marginal expectation of VF as a function of y. 
That is. Ex [IE] = X[u yo)Pr {x = u}. It follows that E [lE] = Ey [Ex [lE] ] . 

Overview of the Analysis. The main steps in the proof of Theorem 1 are as follows. 
In Section 3.1, we show that Ex[E] = Ffe. In Section 3.2, we show that E[Re < 
^2fc^fc Section 3.3, using the above result, we show that Ex [E^] < F^i,. 

Section 3.4 shows that Ey [^2^^] < {2/B + 2^ • and also concludes the 

proof of Theorem 1. Finally, we conclude in Section 4. 



Notation: Multinomial Expansion. Let X be defined as AT = X)i6{o i n-i} 
where, ai > 0, for I G {0, 1, . . . , TV — 1}. Then, can be written as 



X' 



= E 

ei+- 



s—k,ei > 0 ,-' 



,6s >0 



eiC2 • • • e. 



E 

<^2 






where, s is the number of distinct terms in the product and a is the exponent of the 
ji/i pj-oduct term. The indices li are therefore necessarily distinct, k G {0,1,...,TV — 
1}, z = 1, 2, . . . , s. For easy reference, the above equation is written and used in the 
following form. 

^ C(e) ^ ( a:;). (2) 

s,e:Q(e,s) l:i?(e,l,s) J — 1 

where, Q(e, s) = 1 < s < k and e = (ei, 62, • . • , e^) is s-dimensional and = 

k\ i?(e, 1, s) = 1 = {li,l 2 , . . . , Is) is s-dimensional and 0 < li < I 2 <■■■< Is ^ 
1; and the multinomial coefficient <7(0) = ^ g ). In this notation, the following 

inequality holds . 

E “b - (E®i0 • 

l:fl(e,l,s) j=l i=l I 

By setting n = k, and ai = 02 = • • • = flfe = 1, we obtain, 

k^ = J2C{e)Q>J2c{e). 

e,s ^ ' e,s 



By squaring the above equation on both sides, we obtain that k“^^ = ^ C'(g) (g))^ 

> ^ C'^(e). We therefore have the following inequalities. 



e,s 



(4) 
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3.1 Expectation 

In this section, we show that E[Re = Fk, thereby proving Lemma 1, and that 

Ex[r] =Fk. 

Proof (of Lemma 1 ). Since the family of variables x/’s is fc-wise independent, therefore 

E[ . 

i=i i=i 

Applying equation (2) to = (ff2i fixi)^ and using linearity of expectation and fc- 
wise independence property of xi ’s, we obtain 

E[^1= E C'(e) E ( fi-){ EK;]) . 

s,e:Q(e,s) l:i?(e,l,s) J — 1 J — 1 

Using equation (1), we note that the term (IIj=i E[a;^^ ])=0,if s > 1, since in this 
case, Cj < k, for each j = 1, . . . , s. Thus, the above summation reduces to 

E[^1 =E/' = Efc • 

i 

Since Fk is real, E [Re is also Fk, proving Lemma 1. □ 

Lemma 4. Suppose that the family of random variables {a;;} is k-wise independent. 
Then, Ex[n] = Ft,b and^^[v] = E[U] = Fk. 

Proo/ We first show that Ex [n] = Fu,b- fm.bXif] = 

Ex[(Ez hi^bXi)^\ , by letting hi^b = fi • yi,b- By an argument analogous to the proof of 
Lemma 1, we obtain Ex [(E/ hi^bXi)'^] = J2i ^Ib = J2i ftvi,b = J2i ftkVUb = Fk,b, 
(since y/,t,’s are binary variables). Since Fk,b is always real. Ex [Lh] = Ex [Re = 
Fk,b- Finally, Ex[U] = Ex[EbEh] = E6 Ex[L6] = J2b^k,b = Fk, since each ele- 
ment is hashed to exactly one bucket. Further, E [U] = Ey [Ex [E] ] = Ey [pfc] = Fk . 

□ 



3.2 Variance of Re Z'^ 

In this section, we estimate the variance of Re and derive some simple corollaries. 

Lemma 5. LetW = Re btiXiY . Tfcen, Var [lU] < fc^*(Ei®?)^- 

Proof Let AT = (Zi^iXiY- Then, Var[lU] = F.[W‘^] - (E[lU])2 < E[XX] - 
(E [FF] )^. Using equation (2), for X, X, we obtain the following. 

E E ( 

s,e:Q(e,s) l:i?(e,l,s) J — 1 

E E ( xE) 

i,g:Q(g,t) l:R(g,m,t) j' = l i' = l 
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Multiplying the above two equations, we obtain 

E E C'(e).C(g) ^ ^ 

s,e:Q(e,s) i,g:Q(g,i) l:i?(e,l,s) 

<:)■{ ‘ -^)-( ' ^,)- 

j=i j'=i j=i j'=i 

The general form of the product of random variables that arises in the multinomial 
expansion of XX is Since the random variables xi ’s are 2k- 

wise independent, using equation (1), it follows that, 

S t fl if s = f = l,ei = = fc 

E [ ] = \1 ifs = f, f>l,e = g and 1 = m, 

•^=1 7'=i 1^0 otherwise. 

This directly yields the following. 



E[XX]= Y. E 

e,s:Q(e,s) l:i?(e,l,s) j — 1 

s 

<EC7e) (E ) , by equation (3) 

e,5 j—1 I 

^ E (E E - (E 

e,s j=l I I I 

= (E®n^(E‘^^(®))’ since ^ej=fc 

I e,s j=l 

< (Y^ af)^ ■ , by equation (4). □ 

i 

By letting a/ = fi,l € {0, 1, 2 . . . , — 1}, Lemma 5 yields 

Var [Re Z'^] = Var [Re (5) 

i 

which is the statement of Lemma 2. By letting ai = hi^b = fi • yi,b, where, 5 is a hxed 
bucket index, and I G — 1}, yields the following equation. 

Ex[n"] ioibG[B]. (6) 



3.3 Var [ V ] : Vanishing of Cross-Bncket Terms 

We now consider the problem of obtaining an upper bound on Var [Y ] . Note that 
Var[r] = Ey[Ex[r2]] - (Ey [E^ [V] ] )^. ^From Lemma 4, Ey [E^ [V] ] = F^. 
Thus, 

Var[y] =Ey[E4y2]] _^2 ^ 



(7) 
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Lemma 6. Var [F] < [^ 2 b ’ assuming independence assumption I. 



Proof. E^[f 2] = E^[(E,n)2] = MllbYb +lla^byoYb] = Eb'^^Yb] + 

[^aEh] . 

We now consider Ex [Lay{,] , for af^b. Recall that Ya = Re ( and analogously, 
Yb is defined). For any two complex numbers z, w, (Re z)(Re w) = (l/2)Re [z{w + 
tu)). Thus, YaYb = (Re Z„")(Re Z^) = (l/2)Re {Z^Z^ + Z^). 



Let us hrst consider Ex[Z^Z^]. The general term involving product of random 
variables is (0*^1 /f/) • f% ) ' (11^=1 Vh,a • a;®/) • (ri/=i Vm^,,b ■ x%,). Con- 
sider the last two product terms in the above expression, that is, ( 0^=1 Uij,a • ■ 

(X\j' = iyrrLji ,b ■ xJi.,). For any 1 < J < s and 1 < / < t, it is not possible that 
Ij = rriji, that is, the same element whose index is given by Ij = rrij' cannot si- 
multaneously hash to two distinct buckets, a and b (recall that a f- b). By 2fc-wise 
independence, we therefore obtain that the only way the above product term can be non 
zero (i.e., 1) on expectation, is that s = t = I and therefore, e\ = k and gi = k. Thus, 
we have E[Z^Z*^] = = Fu,aFu,b. 



Using the same observation, it can be argued that Ex [Z^Z^ ] = Fk^aFk,b- It fol- 
lows that Ex [(l/2)(Z^Zj -I- Z^Zj ))] = Fk,aFk,b, which is a real number. Therefore 
Ex [Re (1/2)(Z^Z,^ + Z^^)] = Fk^aFk.b = Ex [F„n] • 

By equation (7), Var [F] = Ey [Ex [V^]] — F^. Further, from Lemma 4, F^ = 
Ey[E b Pk,b] ■ We therefore have. 



Var[F] =Ey[Ex[FV(E^M)'] 

b 

= Ey E + E E- - (E 

b a^b b 

= E,EEqrfl+E ^k.,CL^k.,b ^ ^ ^ ^k.,b'^ ] 5 ^boVG 3.r^UlTlCrit 

b a^b b 

= Ey E Ex [n^] - E < Ey [E Ex [n"] ] 

b b b 

< Ey [ y~[ k’^'^F^b] > by equation (6) □ 

b 



3.4 Calculation of E[F|(^] 

Given a f-dimensional vector e = (ei,...,et) such that Cj > 0, for 1 < z < f and 
Ej=i = k, we define the function ip^e) as follows. Without loss of generality, let 
the indices ej be arranged in non-decreasing order. Let r = r(e) denote the largest 
index such that < k/2. Then, we dehne the function (j>{e) as follows. 

V;(e) =nSz=i(i-2eUfc)/^t 

The motivation of this definition stems from its use in the following lemma. 
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Lemma 7. Suppose ^ > 0, for j = 1, ... ,t. Then, n‘ = l F2e, < 

V'(e) • ■ B*. 



Proof. From [1, 2], Fj < v} ^ if j < /c and Fj < F^^ , if j > k. Thus, 



t 


( ^ i 




( r j 


F2ej = 


F2ej 


II 


.^l-2ej/kp2ej/k j 


i=i 









-i2ej /k 



= = ^(e) • • Fl, 



Since 



j=r+l 
^ Cj = k.U 



The function ip satisfies the following property that we use later. 

Lemma 8. If B < 2 • v}~^ , then, ’f{e) < max (fljB, 2^ • vf~'^ jB^'). 

Proof. Let e he a t-dimensional vector. If f = 1, •f{e) = 1/ B.\ft = r, then V'(e) = 
< 2^ • rf-^lB'^. If f > r + 2, then ^’(e) = {2* / B*) ■ 2e^/fc) < 

2‘ • rf-’^jB* < 2^ ■ rf-'^lB^. Finally, let f = r + 1. Then, 

V’(e) = 2* • n*-i-2E < 2* . jB\ 

since X)j=i > r = t — 1. Thus, 

V’(e) < ^(e)(2 • n^-i/B)'^-^ < (2* • n‘-i-^(‘-^)/VB‘)(2 • = 

2fe . ^fe-2-(i-2)/fc/^fc ^ 2^= . . 

where, the first inequality follows from the assumption that B < and the second 

inequality follows because t > 2. □ 



Lemma 9. LetB <2- n^~i. Then, E[F|f,] < k'^F^{2/B + 2^= • / B'^). 



Proof. For a fixed b, the variables yi^ are fc-wise independent. F 2 ,b is a linear function 
of yi b. Thus, F 2 fy is a symmetric multinomial of degree k, as follows. 



F2^ = 



l < 5,6 



• • • f'l^‘yil,b • Vh.b • Vh,b ■ 
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Taking expectations, and using fc-wise independence of the y/,b’s, we have, 

EK.] = E C'(e) E ■ m^,b] 

s,e li<h<-<ls 

= E E fh" ■ ■ ■ ■ ^[vi,A • • -E[yi^,b] 

< 5,6 

= E E AA ■ ■ ■ E [yi.^b] = ^ 

<5,6 

S 

^ E ■ (1/^®) • ^ 2 ej < E ■ Efc^ by Lemma 7 

s,e j — ^ <5,6 

< E C'(e) • -(2/B + 2^- hy Lemma 8 

s,e 

<k'^-Fi-{2/B + 2^-n’^-^/B'=), since, E C'(e) < □ 

s,e 

Combining the result of Lemma 6 with Lemma 9, we obtain the following bound on 
Var [y] . 

Var [r] < . Fl • (2 + 2^= • / B'^~'^) (8) 

Recall that Y is the average of si independent estimators, each calculating Y . The main 
theorem of the paper now follows simply. 

Proof (of Theorem 1). By Chebychev’s inequality, Pr { |F — F^l > eFfe} < 
Var [y] /(e^F^). Substituting Equation (8), we have Var [y] /(e^ • F|) < 1/3. □ 

4 Conclusions 

The paper presents a method for estimating the frequency moment, for k > 2, 
of data streams with general update operations. The algorithm has space complexity 
and is based on constructing random linear combinations using randomly 
chosen k*^ roots of unity. A gap remains between the lower bound for this problem, 
namely, for fc > 2, as proved in [3,5] and the complexity of a known 

algorithm for this problem. 
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Abstract. We study the complexity of computing fc-wise independent 
and e-biased generators G : {0,1}" ^ {0, 1}"*. Specifically, we refer 
to the complexity of computing G explicitly, i.e. given x € {0, 1}" and 
i € {0, l}*°s™^ computing the i-th output bit of G(x). [MNT90] show that 
constant depth circuits of size poly(n) cannot explicitly compute fc-wise 
independent and e-biased generators with seed length n < 2*°® 

In this work we show that DLOGTIME-uniform constant depth circuits 
of size poly(n) with parity gates can explicitly compute fc-wise inde- 
pendent and e-biased generators with seed length n roughly logm ^ 
2*°® In some cases the seed length of our generators is optimal up 
to constant factors, and in general up to polynomial factors. To obtain 
our results, we show a new construction of combinatorial designs, and we 
also show how to compute, in DLOGTIME-uniform ACo, random walks 
of length log“n over certain expander graphs of size 2". 



1 Introduction 

The notion of pseudorandom generators (PRGs) is central to the fields of Gom- 
putational Gomplexity and Gryptography. Informally, a PRG is an efficient de- 
terministic procedure that maps a short seed to a long output, such that certain 
tests are fooled by the PRG, i.e. they cannot distinguish between the output of 
the PRG (over a random input) and the uniform distribution. More formally, a 
PRG G : {0, 1}” ^ (0, 1}™ fools a test M : (0, 1}™ ^ (0, 1} if 

' Pr [M(y) = 1] — Pr [M(G{x)) = 1]' < e. 

One popular class of tests M : {0,1}’” — > {0,1} are parity ones (sometimes 
called linear tests). These are the tests that, given z G {0, 1}’”, only XOR fixed 
subsets of the bits of z. A generator is called e-biased if it fools such tests. 

* Research supported in part by the Leibniz Center, the Israel Foundation of Science, 
a US-Israel Binational research grant, and an EU Information Technologies grant 
(IST-FP5). 

** Research supported by NSF grant CCR-0133096 and US-Israel BSE grant 2002246. 
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Another popular type of tests are those whose value, given z G {0, 1}™, depends 
only on k fixed bits of z, but on those bits is arbitrary. A generator is called 
(e, k)-wise independent if it fools such tests. An important special case of (e, k)- 
wise independent generators are those where e = 0, i.e. every k fixed output 
bits of the generator are uniform over {0,1}^. Such generators are called k- 
wise independent, e-biased and /c-wise independent generators have found several 
applications in Complexity Theory and Cryptography. For a discussion of these 
generators we refer the reader to the excellent book by Goldreich [Gol99]. 

In this paper we study the following general question: what are the mini- 
mal computational resources needed to compute k-wise and e-biased generators? 
Throughout this work, when we refer to the complexity of a generator G, we 
actually refer to the complexity of computing the t-th bit of G{x) given a seed 
X G {0,1}" and an index i G {0,1}*°®’". This is a more refined notion of complex- 
ity, which is especially adequate for generators having logarithmic seed length 
(i.e. n ~ logm), as is the case with /c-wise independent and e-biased generators 
(see below). Generators for which we can efficiently compute the z-th output bit 
of G{x), given x G {0, 1}" and i G {0, l}'°s™^ are called explicit. Explicitness 
plays a crucial role in many applications where only some portion of G{x) is 
needed at each time, and the application runs in time polynomial in the seed 
length of the generator (rather than in its output length). Such applications 
range from constructing hash functions to Probabilistically Gheckable Proofs. 

/c-wise independent and e-biased generators, G : {0, 1}" ^ {0, 1}’", explicitly 
computable in time poly(rz) are known with seed length logarithmic in the output 
length, i.e. n = O(logm) [GG89,ABI86,NN90,AGHP92] (we ignore here other 
parameters, we will be more accurate later). On the other hand, Mansour, Nisan 
and Tiwari [MNT90] showed* that any fc-wise independent or e-biased generator 
G : {0, 1}" ^ {0, 1}’" explicitly computable by constant depth circuits {ACq) of 
size poly(n) must have seed length n at least 2*°®”'^' ’" logm. 

In this paper we ask the following technical question: Gan constant depth 
circuits with parity gates (ACo[0]) of size poly(n) explicitly compute A:- wise 
independent and e-biased generators, G : {0, 1}" ^ {0, 1}’", with seed length 
n « logm? (Where parity is the function ®(a;) := Xi mod 2). 

We give an affirmative answer by showing several constructions of such gener- 
ators explicitly computed by ACo[0] circuits of size poly(n) (our results are dis- 
cussed below). All our circuits are DLOGTIME-uniform. Informally this means 
that each gate (resp. edge) in the circuit can be specified in time linear in 
the name of the gate (resp. edge). DLOGTIME-uniformity is the strongest no- 
tion of uniformity found generally applicable, and gives nice characterizations of 
AGq, AGq[(B] [BIS90]. In this paper whenever we say that a circuit class is uni- 
form we always mean that it is DLOGTIME-uniform. Note that uniform AGo[0] 
is strictly contained in L := logarithmic space. Therefore all our generators are 
in particular explicitly computable in space O(logn). For background on circuit 
complexity we refer the reader to the excellent book by Vollmer [Vol99] . 

* In [MNT90] this negative result is stated for hash functions only, but their techniques 
apply to fc-wise independent and e-biased generators as well. 
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Table 1. Our main results. 



Generators : {0, 1}" ^ {0, 1}”^ explicitly computable 
by DLOGTIME-uniform AGq[©] circuits of size poly(n) 


Seed length 


Type 


Limitations 


Reference 


n = 0{k"' log m) 
n = 0{k^ log(m) log(fc • log m)) 
n = 0(log m 0 log 1 /e) 
n = 0(log m log 1/e) 
n = 0{k 0 log log m 0 log(l/e)) 


fc-wise 
fc-wise 
e-biased 
e-biased 
(e, fc)-wise 


k = 0(1) 

e = G(l/2'°s°‘°e(’"^) 

k = 0(log'=log(m)), 
e= G(l/2'°s°'°g(’")) 


Theorem 2 (1) 
Theorem 2 (2) 
Theorem 4 (1) 
Theorem 4 (2) 
Omitted due to 
space restrictions 



1.1 Our Results 

Our main results are summarized in Table 1. We now discuss some of the gen- 
erators in Table 1. 

k-Wise Independent Generators in ACo[0].' ft-wise independent generators ex- 
plicitly computable in time poly(n) are known with seed length n = 0{k log m) 
[CG89,ABI86] which is optimal up to constant factors (a lower bound of 
(fclogm)/2 was proven in [CGH+85]). 

To understand the difficulty of implementing these generators in ACo[0] 
let us discuss a construction from [GG89,ABI86]. For simplicity of exposition, 
assume that m = 2^ for some integer h. Let GF{2^) be the field of size 2^, then 
for every k, the generator G : {0, 1}” ^ {0, 1}™ defined as 

G(ao,oi,...,Ofe_i)i := where oo, oi, Ofc-i, f G GA(2'*), (1) 

j<k 

is a fc-wise independent generator^ with seed length n = kh = klogm. 

For concreteness, in the following discussion let us fix k at most poly(/i), so 
that poly(n) = poly(/i). To compute the generator in Equation 1 we must first 
find a representation of GF{2^) (e.g., an irreducible polynomial of degree h over 
GF{2)). We do not know how to find a representation of GF{2^) in uniform 
AGo[0] circuits of size poly(/i) for every given h. And even if one has such 
a representation we do not know in general how to compute field operations 
in uniform AGq[ 0]. On the other hand, it is known that for every h of the 
form 2 • 3^ (for some 1) the polynomial 0 1 is irreducible over GF(2) 

(see e.g. [vL99], Theorem 1.1.28). Furthermore, it can be shown that using this 
specific representation of GF(2^) (for /i = 2 • 3*) it is possible to compute field 
operations over GF{2^) in uniform AGq[ 0] circuits of size poly (/i). (Eric Allender 
(personal communication, June 2004) pointed out field multiplication to us. More 
involved techniques seem to give field exponentiation with a slightly growing 

^ This generator outputs non-Boolean random variables. To get Boolean random vari- 
ables we can, say, take the least significant bit of G{x)i. 
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exponent of at most poly(/i)). However, these results are bound to a specific 
field representation, and thus are somewhat unsatisfactory. 

In this work we exhibit alternative constructions of fc-wise independent gen- 
erators computable in uniform HCo[0]. 

Techniques: Our fc-wise independent generators are based on combinatorial 
designs. A combinatorial design is a collection of m subsets Si, , Sm of {1 ... n} 
with small pairwise intersections. We obtain the t-th output bit of the generator 
by taking the parity of the bits of the seed indexed by Si. Roughly speaking, the 
small intersection size of the sets will guarantee the fc-wise independence of the 
variables. For the case k = 0(1), we give a new construction of combinatorial 
designs with n = 0(log m), which is computable in AOo[0] (Lemma 3). Following 
an idea from [HR03] (credited to S. Vadhan), we obtain this construction by 
concatenating Reed-Solomon codes with a design construction from [Vio04] . 

To compute our design constructions (specifically the Reed-Solomon code) 
we work over finite fields as well (as does the classic fc-wise independent generator 
in Equation 1). The difference is that in our constructions we need finite fields 
of size exponentially smaller, i.e. poly(n) (as opposed to 2") . Representations 
of such fields can be found by brute force (even in uniform ACq), and results by 
Agrawal et. al. [AAI+01] show how to perform field operations over fields of size 
poly(n) by uniform ACq circuits of size poly(n). 

The question rises whether there is a construction of /c-wise independent gen- 
erators that does not use finite fields of growing size. We argue (see the remark 
at the end of Section 3) that such a construction is possible using our approach 
based on combinatorial designs, however we only get a generator explicitly com- 
putable in P-uniform ^(70(0] (rather than DLOGTIME-uniform) . 

e- Biased Generators in AC'o[0].‘ Similar to fc-wise independent generators, e- 
biased generators explicitly computable in time poly(n) are known with seed 
length n = 0(log m 0 log 1/e) [NN90,AGHP92] which is optimal up to constant 
factors [AGHP92]. 

Alon et. al. [AGHP92] give three simple constructions, but none of them 
seems to be implementable in uniform ACq [0] : they either seem to be inherently 
sequential, or they need large primes, or they need exponentiation over finite 
fields of size 2" with exponent of n bits, which does not seem to be computable 
in uniform AC'o[0] circuits of size poly(n). 

Prior to their work, Naor and Naor [NN90] gave another construction. We 
show that their construction (as long as e > l/ 2 *° 8 °(iogm) fQj. gome fixed con- 
stant c) can be implemented in uniform ACo[0] (for e < we obtain 

seed length 0((log m)(log 1/e)) with a slight modification of the construction in 
[NN90]). 

Techniques: The construction of [NN90] goes through a few stages. In partic- 
ular it uses as a component a 7-wise independent generator. By the discussion 
above, we have such a generator in uniform ACo[(B]. [NN90] also needs to com- 
pute random walks on an expander graph. We show that, for every fixed c, 
random walks of length log'^ n on the Margulis expander [Mar73] of size 2”, can 
be computed by uniform ACq circuits of size poly(n) (where the depth of the 
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circuit depends on c). We also argue that for this specific expander, our result 
is tight. Bar-Yossef, Goldreich and Wigderson [ZBYW99] show how to compute 
walks on the same expander but in an online model of computation. Their result 
is incomparable to ours. Ajtai [Ajt93] shows how to compute in uniform ACq 
of size poly(n) walks of length log n on expander graphs of size exponentially 
smaller, i.e. n. 

Almost k-Wise Independent Generators in ACo[0].' Using the approach of 
[NN90] that combines /e-wise independent and e-biased generators to get al- 
most fc-wise independent generators, we can use our constructions to obtain 
an (e, /c)-wise independent generator G : {0,1}” ^ {0,1}™ explicitly com- 
putable by uniform AC'o[0] circuits of size poly (n, log m). Its seed length is 
O(fc0loglogm0log(l/e)) as long as k = O(log'^logm) and e = I7(l/2^°s°'°8™) 
(for any fixed constant c > 1). In this range of parameters, this seed length 
matches that of the best known constructions [NN90,AGHP92] up to constant 
factors. Due to space limitations this construction is omitted. 

Organization: In Section 2 we discuss some preliminaries. In Section 3 we show 
the connection between combinatorial designs and fc-wise independent gener- 
ators, and we exhibit our construction of combinatorial designs and our con- 
structions of fc-wise independent generators in uniform ACq [0] . In Section 4 we 
give our e-biased generators in uniform ACq [0] . This includes our results about 
computing walks on expander graphs. 

2 Preliminaries 

We now define fc-wise independent and e-biased generators. Denote the set 
{!,..., m} by [to]. For / C [to] and G{x) G {0,1}™ we denote by G{x)\i G 
{0, 1}I^I the projection of G{x) on the bits specified by I. Recall that 0 is the 
parity function, i.e. 0^^/ ( 2 ;) := E*e/ Xi mod 2. 

Definition 1. Let G : {0, 1}” ^ {0, 1}™ he a generator. 

— G is (e, fc)-wise independent if for every M : {0,1}^ ^ {Oj 1} I U [to] 

such that \I\ < k: | = 1] - Pr^^^o^i}«[M{G{x)\i) = 1]| < e. 

— G is fc-wise independent if it is {Q,k)-wise independent. 

~ G is e-biased if for every I C [to].- | Pr,cg{o,i}» [0,g/ G(x)* = 0] - < e. 

We now define the circuit classes of interest in this paper. ACq is the class 
of constant depth circuits with V and A gates, where V and A have un- 
bounded fan-in. AGq[ 0] is the class of constant depth circuits with -',V,A and 
© gates, where V, A and 0 have unbounded fan-in. A family of circuits {G„} is 
DLOGTIME-uniform, if there is a Turing machine M running in linear time that 
decides the direct connection language 0 / {G„}, which is the language of tuples 
(t, a, 6, n), such that b and a are names of gates in G„, 6 is a child of a, and a is of 
type t [BIS90,Vol99]. In this paper uniform always means DLOGTIME-uniform. 
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Definition 2. A generator G : {0, 1}” ^ {0, 1}'" is explicitly computable by 
uniform ^C'o[0] (resp. ACq) circuits of size g, if there is a uniform ACo[0] 
(resp. ACq) circuit C of size g such that C{x,i) = G{x)i for all r G {0, 1}" and 
i G {0,1}*°®™, where G{r)i is the i-th output hit ofG. 

In [MNT90] they essentially prove the following negative result on the abil- 
ity of AGo circuits to explicitly compute (e, fc)-wise independent and e-biased 
generators. 

Theorem 1 ([MNT90]). Fix any constant e < 1/2. Let G : (0, 1}” ^ (0, 1}™ 
be a generator either {e,2)-wise independent or e-biased. Let G he a circuit of 
size g and depth d such that C(r, i) = G{r)i for every r G {0, 1}”, i G {0, 

Then g > l7(logm). 

In some of our constructions we make use of the following result from 
[AAI+01] about field operations in uniform AC'o[0]. We denote by GF{2*) the 
field with 2* elements, and we identify these elements with bit strings of length t. 

Lemma 1 ([AAI+01], Theorem 3.2 and proof of Theorem 1.1). Let t = 

O(logn). There is a uniform AC'o[0] circuit of size poly(n) such that given a 
polynomial p(x) := degree k = poly(n) over GF{2^), and b G 

GF{2*), computes p{b) . 

3 fc-Wise Independent Generators from Designs 

In this section we present a general approach that gives fc-wise independent 
generators from combinatorial designs. First we define combinatorial designs. 
Then we show how to get fc-wise independent generators from combinatorial 
designs. We then turn to the problem of efficiently constructing designs with 
good parameters. 

Definition 3. [NW94] A {l,d)-design of size m over a universe of size s is a 
family S = {Si, . . . , Sm) of subsets o/ (1, . . . , s| that satisfies: (1) for every i, 
jAil = ?, and (2) for every i j, jS'i n S/| < d. 

We now show how to get /c-wise independent generators from combinatorial 
designs. 

Lemma 2. Let S = {Si,...,Sm) be a {{k + l)d,d)-design of size m 

over a universe of size n. Define the generator Gs : (0, 1}" ^ (0, 1}™ as: 
Gs{'r)i Then Gs is a k-wise independent generator. 

Proof. Fix k output bits i\, . . . ,ik. We show that Gs{x)if, is uniform over (0, 1} 
and independent from Gs(x)ij , . . . , Gs{x)i^,_^ . By definition, Gs{x)i^. is the par- 
ity of the bits in x indexed by S'ij,. Since S' is a {{k 0 l)d, d)-design there is 
e G Si^, such that e ^ Uo<i<fe'^b- Thus the value of Gs{x)i,^ is independent 
from Gs{x)i^, . . . ,Gs{x)i^_i, because its parity includes a bit that is indepen- 
dent from the bits in the parities of Gs(x)i^ , . . . , Gs{x)i^_^ . □ 
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3.1 Combinatorial Designs Computable in Uniform ACo[©] 

We now describe a new construction of combinatorial designs of size m with 
universe size O(logm). In [HR03] (Section 5.3) it was suggested (following a sug- 
gestion of Salil Vadhan) to combine error-correcting codes with combinatorial 
designs to achieve new designs with good parameters. While their construction 
does not achieve the parameters that we need, we can use their idea to obtain de- 
signs with the desired parameters combining Reed-Solomon codes with a design 
construction from [Vio04]. 

Lemma 3. For every constant c > 1 and large enough m there is a family 
S of logm,2clogm)-designs of size m over a universe of size 50 • c^logm. 
Moreover, there is a uniform 7lCo[©] circuit of size poly(logm), such that given 
i G {0 , computes the characteristic vector of Si. 

Lemma 3 uses as a component a family of designs with exponentially smaller 
parameters computable in ACq [Vio04]. 

Lemma 4 ([Vio04]). For every constant c > 1, and for every large enough n 
there is a family S of (c log n, log n) -designs of size n over a universe of size 
50-c^logn. Moreover, there is a uniform ACq circuit of size poly(n), such that 
given i G {0,1}^°®” computes the characteristic vector of Si. 

Proof (of Lemma 3). First we describe the construction, then we show it is a 
design with the claimed parameters and then we study its complexity. 

Construction: The idea is ‘combining’ a Reed-Solomon code with the designs 
given by Lemma 4. Let n be such that nlogn = clogm. Fix a field F of size n 
(without loss of generality we assume there is such a field). Let h := n/c. Let z 
be a bit string z = ... ah where each Oj has log(n) bits. Define the polynomial 

Pz over F as Pz := X)i=o Let 6i, . . . , be an enumeration of all elements 
of F. 

Now consider a family (Z?i, . . . , £)„) of (clogn, logn) designs of size n over a 
universe of size 0(c^ logn) as guaranteed by Lemma 4. 

Then S = {Si , . . . , 52hiogn) is defined as follows: The characteristic vector of 
Sz is Dp^f^hi) ■ ■ ■ -Dp^(b„)- Namely it is a concatenation of n characteristic vectors 
of sets from D. 

Analysis: S has = m sets, each of size cnlogn = c^logm, and the 

universe size is O(c^nlogn) = 0(c® logm). We now bound the intersection size. 
Consider z ^ z' . Since Pz and Pz’ are polynomials of degree at most h, there are 
at most h distinct b GF such that Pz{b) = Pz'{b). Whenever Pz{b) yf Pz'{b), we 
have that Dpz{b) and Dp ,({,) are distinct sets in the design D, and thus their 
intersection is at most log n. Therefore 

I'S'z n S'z' I < hclogn + {n — h) logn < 2nlogn = 2clogni 

Complexity: By Lemma 1, computing Pz{b) can be done by uniform tICq)©] 
circuits of size poly(n) = poly(logm), and D can be computed in uniform ACq 
circuts of size poly(n) = poly(logm) by Lemma 4. □ 
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The construction above gives /c-wise independent generators with k = 0(1) 
(see Theorem 2). For k = u;(l) we use as a component a celebrated design 
construction by Nisan and Wigderson [NW94] which we now state without proof. 
The complexity of computing this construction follows from Lemma 1. 

Lemma 5 ([NW94]). For every integers f , m such that log m < i < m, there 
is a {I, log m)- design of size m over a universe of size 0{£^). Moreover, there is a 
uniform AOo[0] circuit of size poly(£), such that given i G {0,1}*°*™ computes 
the characteristic vector of Si . 

We now state our fc-wise independent generators. 

Theorem 2. For every large enough m, there is a k-wise independent generator 
G : {0, 1}” ^ {0, 1}™, that is explicitly computable by uniform TOo[0] circuits 
of size poly(n), where, 

1. k>2 is a fixed constant, and n = O(fc^logm). Or, 

2. k is any function (of m), and n = 0{kf log(m) log{k ■ logm)). 

Proof. For Item (1) we plug the design construction from Lemma 4 (with c := 
2{k + 1)) into Lemma 2. Item (2) is obtained as follows. Consider a ((fc + 
1) log m, log m)-design S of size m over a universe of size 0{k^ log^ m) as guaran- 
teed by Lemma 5. By Lemma 2, Gs is a fc-wise independent generator, however 
the seed length of G$ is n = 0(fc^ log^ m). To reduce the seed length, suppose we 
have a {k{k + 1) logm)-wise independent generator G' : {0, 1}” ^ {0, 1}”. We 
claim that GoG' : {0, 1}” ^ {0, 1}™ is still a /c-wise independent generator. This 
is because every k output bits of G depend on at most k{k+l) log m output bits of 
G'{x). Since G' is (k{k+l) logm)-wise independent, these bits will by uniformly 
and independently distributed. Now, using for G' the generator of [CG89,ABI86] 
(Section 1.1, Equation 1) we have n' = 0{k- (k+1) log(m) • (log fc0 log (logm)) = 
0(fc^ log(m) log(fc • logm)). Finally, by Lemma 1, G' is computable by uniform 
AGo[0] circuits of size poly(n'). □ 

A “Combinatorial” Construction. We note that our approach allows for a con- 
struction of /c-wise independent generators (for k = 0(1)) that does not use 
finite fields of growing size. This is obtained combining, as in Lemma 3, the 
designs from Lemma 4 with an error correcting code. For the latter, we take an 
expander code [SS96] based on the Margulis expander discussed in Section 4.1. 
The resulting generators match the parameters of Theorem 2 (1), and they are 
computable in P-uniform AGo[0] circuits of size poly(n). 

4 e-Bias in ACo[0] 

In this section we describe our constructions of e-biased generators in uniform 
4Go[0]. We obtain our constructions by exhibiting uniform 4Go[0] implemen- 
tations of the e-biased generator due to Naor and Naor [NN90]. Due to space 
restrictions we do not describe the e-biased generator in [NN90] here. We only 
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point out that it is built combining 7-wise independent generators, 2- wise in- 
dependent generators and random walks on expander graphs. Using our k-wise 
independent generators in uniform ^C'o[0] (Theorem 2), all that is left to do is 
computing walks on expander graphs in uniform ACo[0]. We now formally state 
what random walks we need to get e-biased generators. 

A family of graphs {Gat} at is a family of d-regular expander graphs if there is 
a constant X < d such that for every N the graph Gat has N nodes, is d-regular, 
and the absolute value of the second eigenvalue of its adjacency matrix is at most 
A. To get an e-biased generator we need random walks of length I = 0(log 1/e) 
on expander graphs of size 2*^^”). We show in the next section that for every 
fixed c we can compute walks of length log'^ n on certain expander graphs of size 
2" in uniform ACq circuits of size poly(n). 

Theorem 3. There is a family {Gn}n of 8 -regular expander graphs such that 
for every c there is a uniform ACq circuit of size poly(n) that, given v € G2" 
and a path w of length log'^ n, computes the node v' G G2" reached starting from 
V and walking according to w. 

Using the expander walks in Theorem 3 we get, for e > 1/2*°® an 

e-biased generator with seed length optimal up to constant factors [AGHP92]. 
For smaller e, we replace the random walk on the expander with random (inde- 
pendent) nodes in the graph, and obtain a generator with larger seed. 

Theorem 4. For every large enough m, there is an e-biased generator : 
{ 0 , 1 }" ^ { 0 , 1 }’", explicitly computable by uniform AGq[ 0 ] circuits of size 
poly(n), where 

1. n= 0(logm 0 log(l/e)) and e = i/ 2 '° 8 °(i°s™), for any constant c > 0. Or, 

2. n = 0((logm) log(l/e)) and e = e(m) is arbitrary. 



4.1 Expander Walks in ACq 



In this section we prove Theorem 3, i.e. we show that there is an expander graph 
of size A^ = 2" where random walks of length I = 0(log'^ n) can be computed by 
uniform AGq circuits of size poly(n) (for every fixed constant c; the depth of the 
circuit depends on c). We use an expander construction due to Margulis [Mar73] 
(the needed expansion property was proved later in [GG81,JM87]), which we 
now recall. 

Let m := \fN ( we assume without loss of generality that m is a power of 
2). The vertex set of Gat is Zm x Zm, where Zm is the ring of the integers 
modulo m. Each vertex w is a pair v = (x,y) where x,y G Z^.. For matrices 
Ti,T 2 and vectors 61,62 defined below, each vertex u G Gat is connected to 
T\v, Tiv 0 61, T2U, T 2 V 0 62 and the four inverses of these operations. 



Theorem 5. [Mar73,GG81,JM87] The family {GAr}Ar with fo := 



1 1 
0 1 



n := 



1 0 
1 1 



, 61 := 



and 62 := 



is a family of 8-regular expander graphs (the 



absolute value of the second eigenvalue of the adjacency matrix is 5\/2 < 8). 
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We now prove Theorem 3, with the family {Gn}n of expander graphs from 
Theorem 5. That is, we show that walks of length log'^ n on G2" can be computed 
in uniform ACq circuits of size poly(n). We note that this result is tight, i.e. there 
is no ACo [©] circuit (uniform or not) of size poly(n) that computes random walks 
of length log“^^^ n on G2»*. This is because computing Xi given x G {0, 1}^ 
(which cannot be done in AGq[©] for I = log“^^^ n) can be ACq reduced to 
the problem of computing random walks of length 0{l) over G2« . (Proof sketch: 
Given x, replace ‘1’ in a; with a step along the edge associated with Ti, replace ‘0’ 
in X with a self-loop. Call x' the string thus obtained. Now start at vertex (0, 1), 
and compute a walk according to x' . It is easy to see that the first coordinate of 
the ending node is ’^^Xi). However this negative result relies on the particular 
expander graph and on its representation. We do not know if HGo[©] (or ACq) 
circuits of size poly(n) can compute random walks of length log“^^^ n on some 
expander on w(n) vertices. 

We now turn to the proof of Theorem 3. By definition of Gat, there are 
8 matrices Ti , . . . , Tg with constant size entries and 8 vectors &i , . . . , 6g with 
constant size entries such that the set of neighbors of a vertex v are {TiV + hi : 
t < 8}. Thus computing the random walk translates to computing 

v' = Ai(. . . (^13(^2(24.11; -|- oi) -|- 02) -l- 03 ) . . .) -I- a; 
where for every i, Ai G {Ti : i < 8} and Oi G {hi : i < 8}. We write this as 
v' = Av+A' where A\= Ai . . . A 2 A 1 and A' ■= Ai .. . ^201-1-21 / . . . A 3 G 2 + . . -+ai. 

So we are left with the following tasks: computing the matrixes 21, 2I' and then 
computing Av + A' . We now show how to solve these problems in uniform ACq. 
First note that since the matrixes 2li’s have constant size entries, the matrixes 
A and A' have entries of size at most 0(1) = log'^ n bits for some c. 

To compute Av + A! given 21, 2I' and v we use the facts that (1) sum of two 
n-bit integers is in ACq and (2) multiplication of a n-bit integer by a log° n-bit 
integer is in ACq. For (1) see e.g. [Vol99], Theorem 1.20, for (2) see e.g. [Vol99], 
Theorem 1.21 (this latter theorem shows multiplication of a n-bit integer by 
a log n-bit integer. The same techniques give (2)). These circuits can be easily 
shown to be uniform. 

We now show how to compute 2I, the same techniques give 2I'. 

Lemma 6. For every fixed constant c there is a uniform ACq circuit of size 
poly(n) that, given I = log'^n, 2x2 matrixes Ai,...,Ai with constant size 
entries, computes A = Hi</ 

Proof. Instead of proving the lemma directly, it is convenient to show that the 
product of n (as opposed to 1) given matrices Ai, . . . , An with constant size 
entries can be computed in space O(logn), and then appeal to the following 
lemma (that can be obtained combining results in [Nep70] and in [BIS90], details 
omitted). 
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Lemma 7 ([Nep70] + [BIS90]). Let L he a language computable in logarith- 
mic space. Then for every constant c > 1 there is a uniform ACq circuit of size 
poly(n) that, given x of size log'^n, correctly decides whether x is in L. 

Suppose we are given n matrices Ai, , A„ with constant size entries. The 
idea is to first compute rii<n in Chinese Remainder Representation (CRR) 
and then convert the CRR to binary (using a result by Chiu, Davida and Litow 
[CDLOl]). More formally, note that every entry of rii<fc ^^r every k < n, will 
be at most d", for some constant d. By the Chinese Remainder Theorem each 
number a; < d" is uniquely determined by its residues modulo poly(n) primes, 
each of length O(logn). The Prime Number Theorem guarantees that there will 
be more than enough primes of that length. We refer to such a representation of 
a number x (i.e. as a list of residues modulo primes) as the CRR of x. Note that 
to find the primes for the CRR we search among the integers of size O(logn). 
This clearly can be done in logarithmic space. 

We compute the product of the n matrices modulo one prime at a time, 
reusing the same space for different primes. To compute the product modulo 
one prime first note that transforming a matrix in CRR is easy because the 
matrixes have constant size entries. Each matrix multiplication is a simple sum 
(because the matrices have constant size entries) and therefore can be computed 
in logarithmic space. The machine operates in space 0(log n) because it only 
needs to store a constant number of residues modulo a prime of length 0(log n). 

All that is left to do is to convert the product matrix from CRR to binary, 
and this can be done in space O(logn) by a result in [CDLOl]. □ 
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Abstract. We consider distribution-free property-testing of graph con- 
nectivity. In this setting of property testing, the distance between func- 
tions is measured with respect to a fixed but unknown distribution D on 
the domain, and the testing algorithms have an oracle access to random 
sampling from the domain according to this distribution D. This no- 
tion of distribution-free testing was previously defined, and testers were 
shown for very few properties. However, no distribution-free property 
testing algorithm was known for any graph property. 

We present the first distribution-free testing algorithms for one of the 
central properties in this area - graph connectivity (specifically, the prob- 
lem is mainly interesting in the case of sparse graphs). We introduce three 
testing models for sparse graphs: (1) a model for bounded-degree graphs, 
(2) a model for graphs with a bound on the total number of edges (both 
models were already considered in the context of uniform distribution 
testing), and (3) a model which is a combination of the two previous 
testing models; i.e., bounded-degree graphs with a bound on the total 
number of edges. We prove that connectivity can be tested in each of 
these testing models, in a distribution-free manner, using a number of 
queries independent of the size of the graph. This is done by providing 
a new analysis to previously known connectivity testers (from “stan- 
dard”, uniform distribution property-testing) and by introducing some 
new testers. 



1 Introduction 

The classical notion of decision problems requires an algorithm to distinguish 
objects having some property V from those objects which do not have the prop- 
erty. Property testing is a relaxation of decision problems, where algorithms are 
only required to distinguish objects having the property V from those which are 
at least “e-far” from every such object. The main goal of property testing is to 
avoid “reading” the whole object (which requires complexity at least linear in 
the size of its representation); i.e., to make the decision by reading a small (pos- 
sibly, selected at random) fraction of the input (e.g., a fraction of size polynomial 
in 1/e and poly- logarithmic in the size of the representation) and still having a 
good (say, at least 2/3) probability of success. 

The notion of property testing was introduced by Rubinfeld and Sudan [19] 
and since then attracted a considerable amount of attention. Property testing 
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algorithms (or testers) were introduced for problems in graph theory (e.g. [1,9, 
11,17]), monotonicity testing (e.g. [5,8,14]) and other properties (e.g. [2,4,16]; 
the reader is referred to surveys by Ron [18], Goldreich [7], and Fischer [6] for a 
presentation of some of this work, including some connections between property 
testing and other areas). In these papers, the distance between an object and 
the property is measured by the fraction of the domain that has to be modified 
in order for the object to obtain the property. For example, when dealing with 
graphs, this distance is measured by the number of edges that should be added 
or removed so that the given graph posses the property. 

However, it is natural to consider situations where not all edges are equiv- 
alent. That is, removal or addition of some edges may be more expensive than 
others, depending on various factors of the problem. We are interested in taking 
these differences between the edges into consideration when measuring the dis- 
tance between a given graph and the property. In other words, we wish to put 
different weights (probabilities) on different elements of the domain. This notion 
of testing, termed distribution-free testing, has been previously introduced by [9] 
and some negative results for distribution-free testing of certain graph proper- 
ties were proved (see below); In [13] the authors presented the first (non-trivial) 
distribution-free testers for two properties: monotonicity and low-degree poly- 
nomials. This raises the question whether distribution-free testers can be found 
for problems in other areas, and specifically graph properties, where the concept 
of non-uniform weights for the points of the domain seems natural. 

In this paper, we study distribution-free testing of graph properties, and 
more specifically the connectivity property. In this case, we wish to distinguish, 
with high probability, connected graphs from those that are far from any such 
graph. The distance between two graphs is measured with respect to some fixed 
but unknown distribution D. Testing of graph properties in general, and con- 
nectivity in particular, are among the most central problems studied in the field 
of property testing (with respect to the uniform distribution). Below we survey 
some of the relevant previous work. 



Graph Representations and Previous Results: In the study of graph properties 
(e.g. [1,3, 9-12, 15, 17]), most previous work distinguish between two basic mod- 
els - one is more suitable for dense graphs while the other is more suitable for 
sparse graphs. In the first model, a graph is represented by its adjacency matrix 
or by a function f : V x V ^ {0, 1}, specifying for every u,v G V whether there 
is an edge between u and v. The distance between two graphs in this model, 
is measured by the fraction of entries in the matrix that should be altered in 
order to transform one graph to the other. This model was first considered (in 
the context of property-testing) by Goldreich, Goldwasser and Ron in [9], who 
proved that a variety of graph-partition problems can be tested with query com- 
plexity poly{^) (which is independent of the size of the graph). Testing in this 
graph model was further studied, for example, in [1], where a characterization 
is given of the first order properties that are testable using a number of queries 
independent of n. 
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Though representation as an adjacency matrix seems very natural, in the 
context of testing it is only relevant when dealing with dense graphs; that is, 
graphs whose number of edges is I7(n^). This is because every two sparse graphs 
(that contain o(n^) edges) are close in the adjacency matrix representation. 
Hence, another model is needed to deal with sparse graphs. Property-testing 
for sparse graphs was first considered by Goldreich and Ron [11], who defined 
the following representation for hounded- degree graphs. Let G = (V, E) be an 
undirected graph with bounded-degree d (i.e., every vertex in V has at most d 
neighbors) and assume that, for every vertex v G V, there is an ordering among 
the neighbors of v. The graph G is viewed as a function f : V x{l,...d}^yu 
{T} where f{v, i) is set to be u if the neighbor of v is u, and is set to be T if u 
has less than i neighbors. Notice that, since the ordering of the neighbors of each 
vertex is arbitrary, a graph G may have more than one function representing it. In 
this representation, the distance between two graphs is measured by the fraction 
of / values that have to be changed in order to transform one function into the 
other. [11] shows, for example, that in this graph representation, connectivity, 
k-edge connectivity, fc- vertex connectivity and cycle freeness are testable using a 
constant (i.e., a function of e~^, but independent of n and d) number of queries, 
and that planarity is testable using 0(d^ • e~^) queries. Properties of graphs 
represented as incidence lists of length d have also been studied in [3] and [10]. 

A different representation was presented by Parnas and Ron [17] for testing 
properties of all sparse graphs, rather than only the bounded-degree ones. In 
this case, the graphs are represented by incidence lists of varying length. For 
each vertex v the tester can query both the degree of v and the i'th neighbor 
of V. The distance between two graphs is then measured by the fraction of edge 
modifications necessary to obtain the property, defined with respect to an upper 
bound m on the number of edges in the graph. They present a testing algorithm 
for graph diameters, in this representation. In addition, they explained how many 
of the results proved in [11] for the bounded-degree case, can be transformed to 
this representation. Among these properties are the testing algorithms for k- 
connectivity. This graph representation was further studied in [15], where the 
testing model is adjusted to deal with dense graphs as well, by allowing the tester 
to query whether there exists an edge in the graph between two given vertices. 
The authors study testing of graph bipartiteness in this model. 

Connectivity Testing: Connectivity is a central property in sparse graphs; as 
mentioned above, it has been previously studied in the context of testing with 
respect to the uniform distribution. However, although extending the testing 
models for graph problems to the distribution-free setting seems natural, the 
problem of connectivity testing in the distribution-free setting has not been dealt 
with before. Moreover, as mentioned, the only known results for distribution-free 
testing of graph properties in general are impossibility results for graphs repre- 
sented by adjacency matrix [9]. They proved that it is impossible to test a variety 
of partition problems (for which they showed testers with respect to the uniform 
distribution) in a distribution-free manner. The generalization of the adjacency 
matrix model to a distribution- free one, is straightforward; on the other hand. 
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every graph is close to be connected in this representation with respect to any 
distribution^, hence testing connectivity is trivial. However, unlike the dense 
case, when dealing with representations for sparse graphs, their generalization 
to the distribution-free case is not so straightforward, and the testing is not 
obvious. 

Our Contributions. We consider three representations of sparse graphs in the 
distribution-free setting. These representations are generalizations of the ones 
presented previously for sparse graphs in the context of uniform distribution 
testing. We show that in each of these representations, connectivity can be tested 
in a distribution- free manner, using a number of queries independent of the size 
of the graph. 

~ The first representation is for bounded-degree graphs [11]. We show that it is 
possible to test graph connectivity, in this representation, in a distribution- 
free manner, using a number of queries which is polynomial in i. The test 
is similar to the one presented in [11] for the uniform setting, however the 
analysis required for the distribution- free case is different (Section 3). 

— The second representation is for graphs with a bound m on the total number 
of edges [17]. We show that in this representation, it is possible to test 
connectivity, in a distribution-free manner, whenever m > n(l -I- c) for some 
constant c using a poly(i) number of queries (Subsection 4.1) . 

— We then combine the two previous testing models, and deal with bounded- 
degree graphs with at most m edges. This model was not studied before 
even in the uniform setting. Hence, we first present a uniform distribution 
tester for this model for every m; then, using the distribution-free tester for 
the previous model, we prove that there exists a distribution-free tester for 
graph connectivity in this model whenever m > n(l -I- c), for some constant 
c (Subsection 4.2). 

2 Definitions 

In this section, we define the notions of being e-far from a property V with 
respect to a given distribution D, and of distribution-free testing. Denote the 
range of functions in question by A, and denote by [n] the set {1, . . . , n}. 

Definition 1. Let D he a distribution over a set X. The D-distance between 
functions f,g:X^Ais distoU^g) = yf g(x)}. 

The D-distance of a function f from a property V (i.e., the class of functions 
satisfying the property V) is distoif ,V) = min^gp distnif^g). We say that f is 
(e,I?)-far from a property V if distu{f,V) > e. 

When the distribution in question is the uniform distribution over X, we 
either use U instead of D or (if clear from the context) we omit any reference to 
the distribution. 

^ Under any distribution measure on U x U, the total probability of the path with 
minimal probability is at most -1. 
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Next, we define the notion of distribution-free tester for a property V . Note 
that the type of queries possible for the tester is determined by the model (as is 
the case in the literature on testing with respect to the uniform distribution). 

Definition 2. A distribution-free tester for a property V is a probabilistic oracle 
machine M , which is given a distance parameter e > 0, and an oracle access to 
an arbitrary function f \ X ^ A and to sampling of a fixed but unknown 
distribution D over X, and satisfies the following two conditions: 

1. If f satisfies V, then Pr{M^’^ = Accept} = 1. 

2. If f is (e, D)-far from V, then Pr{M^’^ = Accept} < 

Note that a more general definition of testers that allows two-sided errors is not 
needed here; all our testers, like many previously-known testers, have one-sided 
error and always accept any function that satisfies the property V in question. 

The definition of a uniform distribution tester for a property V can be derived 
from the above definition by omitting the sampling oracle (since the tester can 
sample in the uniform distribution by itself) and by measuring the distance with 
respect to the uniform distribution. 

Finally, notice that since the distribution D in question is arbitrary, it is 
possible that there are two distinct functions / and g such that distnif, g) = 0. 
Moreover, it is possible that f ^ V and g & V. Since the notion of testing is 
meant to be a relaxation of the notion of decision problems, it is required that the 
algorithm accepts functions that satisfy V, but may reject functions that have 
distance 0 from V and do not satisfy V . In addition, note that the algorithm is 
allowed to query the value of the input function also in points with probability 
0 (which is also the case with membership queries in learning theory)^. This 
definition of distribution-free testing was introduced in [9, Definition 2.1]. 

3 Connectivity Testing of Bounded-Degree Graphs 

The representation for bounded-degree graphs which is the subject of this section 
is a generalization of [11] to deal with arbitrary probability distributions. First, 
we generalize the notion of a function fa'-Vx[d]^VU{J-} that represents 
a degree d (undirected) graph G. Since the d outgoing edges of a vertex v may 
have different probabilities, it is essential to allow fa{v,i) to equal T even if 
fa{v,i + 1) yf T (i.e., we cannot assume that a node of out-degree d' < d uses 
the first d' entries in its incidence list). 

Definition 3. A function fa '■Vx[d\ ^ V U{T} represents a graph G = (V, E), 
if the following holds: 

(a) for every edge (u,v) G E there exist unique z„ and iy such that /g(w,Zu) = v 
omd foivAv) = u, and 

(b ) for every vertex v G V and i G [d] , if there exists no neighbor u of v such 
that fdvji) = u, then /g(w,z) = T. In this case, we say that the pair (v,i) is 
free in fa- 

^ It is not known whether membership queries are essential in general for testing even 
in the uniform case (see [18]); such a result is known only for specific problems such 
as monotonicity testing (see [5]). 
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Denote by the class of functions / that represent connected graphs with 
bounded-degree d. Given a probability measure D : V x [d\ ^ [0, 1] and a 
function fa that represents a graph G, we are interested in the D-distance of fa 
from Hence, we examine possible ways to transform fa into a function fa' 
that represents a connected graph G' with degree d (i.e., fa' G V‘^). To do so, 
we have to connect all the connected components of G; that is, if the connected 
components of G are Gi, . . . ,Ck, then we wish to add an edge between Gi and 
Gi+i while, at the same time, preventing the graph’s degree from exceeding d, 
and keeping the connectivity of each of the connected components. In addition, 
we want the total probability of the modified (i.e., added and removed) edges to 
remain small. Therefore, we look, in any connected component Gi of G, either 
for an edge that is unnecessary to the connectivity, or for two distinct free pairs 
{v,iv) and (u,iu)- In other words, we find a list of possible connecting points 
between the connected components of G. We define more accurately the notion 
of such a list. For this purpose, we need the following definition. 

Definition 4. Let G = (T, E) be a graph, let e = (u,v) G E be an edge, and let 
G be a connected component of G containing e. We say that e is redundant, if 
removing e from E does not affect the connectivity of C. 

The following definition formally states the idea of a list of possible connec- 
tions between the connected components of a graph. The m'th edge in this list, 
will connect the components Gm and Cm+i- 

Definition 5. Let fa be a function that represents a graph G with bounded- 
degree d, and let G\,...,Ck be the connected components of G. We say that a 
list of k quad-tuples L= ((mi, ii), (vi, ji)), • ■ • , ((ufc, *fc), (vfc, jfc)) connects fa if 
the following holds, for every m G [k]: 

1. Um G Gm and Vm G Gm+i (in case m = k, then m-\- 1 refers to 1 ). 

jiri) (nm+lGm+l)- 

3. one of the following holds: (a) (um+i,im-i-i) and {vm,jm) are free in fa; or 
(b) {um-i-i,Vm) is redundant in Cm+i, and /G(Mm-i-iDm-i-i) = Vm, fG{vm,jm) = 
nm+i 

Indeed, given a list L that connects fa, we can construct from fa a function 
fa', that represents a connected degree d graph G', by setting fa'{um,im) = 
Vm and fa'(vm, jm) = Um (notice that the graph G' is obtained from G by 
removing at most one redundant edge from each connected component, thereby 
not damaging the component’s connectivity). 

Let fa be a function that represents a graph G. For every edge (u, v) G 
E, define the D -probability of the edge (u,v) to be the total probability of its 
endpoints with respect to D (it can be seen as the cost of a change in the 
edge (u,v), since a change in the edge (u,v) requires a change in the incidence 
lists of both u and v); equivalently, let and iy be such that fa{u,iu) = 

® The above definition actually describes a cycle connecting all the connected compo- 
nents of the graph. It is possible to define a path from Gi to Ck', however, in such 
a case, not all the quad-tuples are symmetric, causing the definition to be slightly 
more complicated. 
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V and fa{v,iy) = u, then the D-probability of the edge (u,v) is D{u,v) =* 
D{u,iu) + D{v,iy). In addition, define the ^-probability of a vertex v & V 
to be D{v) = Z'ig[d]Z?(v, z) and the D-probability of a list L to be D(L) = 
EmiD{ ) “t“ D(Vfn^ jm)) • 

Observation 1: If a list L connects fa, then the D-distance of fc from is 
at most D{L). 

We present an 0(e“^) distribution-free tester for connectivity of bounded- 
degree graphs that, given access to random sampling of F x [d] according to the 
distribution D and to membership queries of a function fc, distinguishes, with 
probability at least |, between the case that fc represents a connected graph 
with bounded-degree d (i.e., fc G V^), and the case that fc is (e, Z?)-far from 

For lack of space, we deal here only with the case where d is at least 3. An 
algorithm for the special case d = 2 can be constructed using similar arguments 
and will appear in the full version of the paper. 

The tester is similar to the one presented in [11] for the uniform distribution, 
in the sense that it also looks for small connected components in G. However, 
while the original analysis is based on the fact that the number of small connected 
components in a graph G' which is far from being connected, is big, this claim 
no longer holds when dealing with arbitrary distributions^. Hence, a whole new 
analysis is required for the distribution-free case. A natural generalization of the 
tester for the uniform case may seem to be seeking for connected components 
where the total probability of their vertices is small (note that, there is no 
correlation between the number of vertices in a connected component and their 
total probability). However, there are some drawbacks to this approach. First of 
all, in the distribution-free setting we have no knowledge of the actual probability 
D of the sampled points, and are only allowed to sample the domain according to 
D; hence, we are only able to estimate their probability. In addition, the size of 
such components may be very large, therefore finding out whether two vertices 
lie in the same component may not be possible in time independent of the size 
of the graph. Thus, a different generalization is required. 

Algorithm - connectivity(e, d) 

Repeat | times: 

— Choose, using the sampling oracle, (v, i) ~ D. 

— Perform BFS starting from v until ^ vertices have been reached, or no 
new vertex can be reached. If the search was ended since no new vertex 
can be reached, return FAIL®. 

return PASS 

^ Consider for example a graph G that consists of two connected components, one of 
size |F| — 1 and the other containing a single vertex vo, and the distribution D is 
set to be D{vo,i) ~ ^ for every 1 < i < d. In this case, G is (^,D)-far from being 
connected, while it contains only one small connected component 
® Assume |F| > If|F| < fj, then it is possible to decide whether the graph is 
connected in query complexity 0{e~^). 
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To prove the correctness of this algorithm, we use the following notation. Let 
fc be a function that represents a graph G, and let G be a connected component 
of G; denote by nc the number of vertices in G and define wc = D{v) 

(i.e., the total probability of all the vertices in G). 

The next lemma states that if fa is (e, D)-far from then the total weight 
of the small connected components in the graph G is at least Denote by 
Suifa) the total probability of connected components in G whose size is smaller 
than i.e., Soifa) = J2c-.nc<^^C- 

Lemma 1. Let fa be a function {e, D)- far from . Then, Saif a) > §• 

To prove the lemma, we show that if Soifc) < f > then it is possible to construct 
a list L that connects fa, such that D{L) < e. Then, by Observation 1, we 
deduce that distnifaj'P^^) < e- The full proof is omitted and will appear in the 
full version of the paper. 

We can now prove the correctness of the algorithm connectivity. 

Theorem 1. Algorithm connectivity {e, d) is a distribution-free tester for con- 
nectivity of bounded- degree graphs with degree d > 3; its query complexity is 
0(e-2). 

Proof. By the definition of a function representing a graph, in every stage of 
the BFS the algorithm looks for the next neighbor of the current vertex, which 
causes the BFS step to cost 0{d) queries (in the worst case). Since there are 
O(^) BFS steps, the query complexity is as required. By the definition of the 
algorithm, if fa G V’^, then it is accepted by the algorithm with probability 1. Let 
fa be (e, D)-far from T”^. By Lemma 1, Saif a) > f- Hence, the probability of 
the algorithm to randomly pick a pair {v, i) such that v G C and G is a connected 
component of size less than is at least |. Therefore, the probability that the 

4 1 2 1 

algorithm fails to find a small component is at most (1 — < 5 . □ 

4 Alternative Testing Models for Sparse Graphs 

In this section, we consider additional testing models for sparse graphs, as dis- 
cussed in the Introduction. For each of these testing models, we define the class 
of functions that represent graphs, the queries that the tester is allowed to ask, 
and prove the existence of a distribution- free tester in that model. Section 4.1 
deals with graphs with a bounded number of edges. Then, Section 4.2 deals with 
bounded-degree graphs with a bound also on the total number of edges. 

4.1 Graphs with a Bound on the Number of Edges 

The testing model described in this section is a generalization of the model 
presented in [17] for testing with respect to the uniform distribution: There is no 
upper bound on the degree, only a bound m on the total number of edges in the 
graph; as in the uniform model, the tester can query the degree of a given vertex 
V or the i'th neighbor of v. As explained in [17], connectivity can be tested with 
respect to the uniform distribution. 
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To generalize this model to the distribution-free setting, we view a graph 
G = (V,E) as a function / : [m] (^)U{-L}i indicating for each i e [m] the 

i'th entry in the edge list if such an edge exists, or _L if that entry is empty. 
As before, a graph G can have more than one representation, due to different 
orderings of the edge list. Denote by Vm the class of functions that represent 
connected graphs with at most m edges. 

Note that, the differences between the weights of the entries in the list are 
more significant when removing edges (in order to make room for new edges) 
and less relevant when talking about edges additions (where one can choose the 
cheapest entry). 

The next lemma, shows that if m > (1 -I- c)n for some constant c, then it is 
possible to use any tester known in this model for the uniform distribution, to 
construct a distribution-free tester. 

Lemma 2. Let m > (1 + c)n, for some constant c. Then, there exists a distri- 
bution-free tester for Vm with query complexity ofO{e~^). 

To prove the lemma, we show that if fa is e-far from Vm with respect to any 
distribution D, then it is also g(e)-far from Vm with respect to the uniform 
distribution, when g{e) is a linear function in e. (proof omitted.) 

4.2 Graphs with a Bound on the Number of Edges 
and on the Degree 

Next, we consider a combination of the two previous testing models. One may 
think of graphs where there are at most m edges and the degree of each vertex 
is at most d. This model can be seen as an intermediate model between the 
above two models, since it contains all bounded-degree graphs and is contained 
in the set of all graphs with a bound on the total number of edges. As in the 
previous model, graphs are represented by functions / : [m] ^ (^)U{-*-}) &nd 
the distribution is defined over the list of edges, i.e. the set [m]. However, in this 
case we are only interested in functions that represent degree-d graphs with at 
most m edges. Denote the class of these functions by Vf^. As in the previous 
model, the tester is able to query the i'th neighbor of a given vertex v. 

Though we believe that this is a very natural testing model for sparse graphs, 
it was not considered before even in the uniform setting. We prove existence 
of both uniform and distribution-free testers for Vm, using similar approaches. 
When dealing with the uniform distribution, the distance between two functions 
is measured as in [17]; that is, by the fraction of edge modifications necessary to 
transform the graph into a connected one, measured with respect to m. 
Lemma 3. There exists a uniform tester for V^ with query complexity poly 

To prove the above lemma, we show that if a function fa is e-close to Vm, 
then it is 3e-close to Vm (note that both testing models use the same graph 
representation). It follows, that if fa is e-far from Vm, h is §-far from Vm- 
Hence, the existence of a uniform tester for Vf;,^ follows from the existence of 
such a tester for Vm [17]- The full details are omitted for lack of space. 

We now turn to the distribution-free setting. Assume that d > 3; the proof 
for d = 2 is similar. 
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Theorem 2. Let m > (l + c)n, for some constant c. There exists a distribution- 
free tester for with query complexity poly{^, d). 

To construct the distribution- free tester, we want to use the tester presented for 
bounded-degree graphs in Section 3. Hence, we first need a way to transform 
graphs in our representation to the bounded-degree representation. We describe 
how, given a function fc : [m] (^)U{-L} that represents a graph G with 

degree at most d, we transform it to a function T{fc) : H x [d] — > HlJ{-L} 
that represents the same graph G. The transformation is as follows: for every 
V G V, assume an ordering on the edges outgoing from v (i.e., the same ordering 
according to which the tester’s queries are answered). For every i G [d\, define 
the value of T{fa)(v, i) = uifuis the i'th neighbor of v and _L if z; has less than 
i neighbors. Note that necessarily if T{f){v,i) = -L, then T{f){v,i -I- 1) = _L. 

Note that, given access to queries for the function fa, one can simulate 
queries for the function T^fc) (this is due to the choice of the same ordering on 
the outgoing edges of a vertex v). 

However, in order to use the distribution- free tester for bounded-degree 
graphs, we need to also find a way to translate distribution measures from our 
model to the bounded-degree model. 

Given the functions fa and T^fc), we describe a transformation from the 
distribution D defined over [m] to a distribution T{D,fc) defined over V x[d\. 
Define W to be the total probability of all non-empty entries in [m] under the 
distribution D. That is, W = The distribution T{D,fQ) is now 

defined as follows. For every v gV and z G [d]: 

• If T{fc){v, i) = u and (u, u) is the j'th edge in the list, define T{D, /g)(w, z) = 
D{j)/2W. If IF = 0, then T^D^fc) is defined to be the uniform distribution 
over all non-empty entries in [to] . 

• Otherwise, define T(D,/g)(u,z) = 0. 

Note that, sampling of F x [d] with respect to the distribution T{D, fa) using 
sampling of [to] according to the distribution D is not possible. This is due to 
the possibility to sample empty entries in [to], that cannot be translated to a 
pair {v,i) in F x [d]. 

Sampling of T{D, fa) : Set c = 8. 

• Sample, at most — log - times, the distribution D until the sampled value z is 
such that /(z) yf _L. 

• If all — log - samples were of empty entries, return FAIL. 

• Otherwise, let z be the sampled non-empty entry, and assume /(z) = (v,u). 
Find the location of u in u’s neighbors list, denote it by z„. Similarly, find z;’s 
place in u’s neighbors list, denoted by z„. Then, with probability ^ return {v, iy), 
otherwise return (zz, z„). 

Each sampling of T{D,fc), using the above sampling procedure, requires 
poly{^,d) queries. In addition, if the sampling procedure indeed returns a value 
(v,i), then (v,i) ~ T{D,fG)- However, there is also the possibility that the 
sampling procedure returns FAIL, which does not happen in direct sampling 
of T{D,fa)- Let A be a connectivity distribution-free testing algorithm in the 




Distribution-Free Connectivity Testing 403 



bounded-degree model. Denote by A' the algorithm obtained from A by adding 
the following rule: if any sampling query of A, during its run on a function /, 
was answered FAIL, then A accepts /. We will show that we can still use the 
tester after this modification for our construction. 

Assume for now, that we actually have an ability to sample T{D,fc)- One 
may wonder whether the distribution- free tester for that was presented in 
Section 3, can be used as a tester for as well. It is easy to see, however, that in 
some cases the function T{fa) is very close to with respect to T{D, fc), while 
fa is far from with respect to D Therefore, a different test is required. 
The distribution-free tester for is as follows: 

Algorithm Connectivitym{e,d): 

Repeat twice: 

Run connectivity’ (e, d) (obtained from the algorithm presented in 
Section 3) on T(fa) using the sampling procedure for T{D, fa) with 
distance |. 

Run the distribution-free tester for Vm on fa and D with distance |. 

return PASS 

Theorem 2 now follows immediately from the correctness of the above tester. 
This, in turn, is based on Lemma 4 and Lemma 5 below, and the existence of 
the distribution-free testers for Vm (Section 4.1). As in Section 3, we denote 
by Soif) the total weight of the small connected components in G. That is, 
the total weight of connected components of size less than ||. Lemma 4 below 
shows that if the total weight of small connected components in G, measured 
with respect to D, is large then, with high probability (over the possible runs 
of the tester), the function fc fails to pass the first stage of the above tester. 
Lemma 5 then shows that if the total weight of small connected components in 
G is small, and the function is far from Vm, then with high probability, over the 
possible runs of the tester, it fails to pass the second stage of the above tester. 

Lemma 4. Let fc : [m] (^)U{-L} a function and D a distribution over 

[m\. If Su{fc) > |> then the probability that fc passes the first stage of the 
tester is at most |. 

The proof of this Lemma uses the following observation, which is based on the 
analysis of the tester for (Section 3). 

Observation 2: Let fc be a function in the bounded-degree model. If Soifa) > 
|, then the probability that / passes the distribution-free test presented in Sec- 
tion 3, with respect to a distribution D over P x [d] with distance parameter e, 
is at most 



Let m = nd(A, let D be the uniform distribution over [m], and consider the following 
graph G on n nodes. G consists of n/2 isolated vertices and a connected component 
of size n/2 that contains nd/4 edges. It is easy to see that distoifGjVm) > 1/d, 
while distT{D ,fc)(I' U g) ,V^) = 4/nd. 
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Lemma 5. Let f : [to] ^ (^)U{-L} be a function and D a distribution over 
[to]. If f is ^-close to Vm with respect to D, and Soifa) < then f is e-close 
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Abstract. A fc-uniform hypergraph G of size n is said to be e-far from 
having an independent set of size pn if one must remove at least en^ edges 
of G in order for the remaining hypergraph to have an independent set of 
size pn. In this work, we present a natural property testing algorithm that 
distinguishes between hypergraphs which have an independent set of size 
> pn and hypergraphs which are e-far from having an independent set of 

2k 

size pn. Our algorithm is natural in the sense that we sample ~ c{k)^-^ 
random vertices of G, and according to the independence number of the 
hypergraph induced by this sample, we distinguish between the two cases 
above. Here c{k) depends on k alone {e.g. the sample size is independent 
of n). To the best of our knowledge, property testing of the independence 
number of hypergraphs has not been addressed in the past. 



1 Introduction 

A /c-uniform hypergraph is a hypergraph G = (V, E) in which each (hyper) edge 
is of size exactly k. An independent set I in G, is a subset of vertices that do not 
include any edges {i.e. there does not exist an edge {vi, ... ,Vk} € E for which 
Vi £ I for all i G {1, . . . , A:}). The size of the maximum independent set in G is 
denoted by a{G), and referred to as the independence number of G. Consider a 
fc- uniform hypergraph G of (vertex) size n, with a maximum independent set / 
of size a{G). Let iL be a random subgraph of G of size s {i.e. H is the subgraph 
induced by a random subset of vertices of G of size s). In this work we study 
the independence number of El. 

For example, if it is the case that a{G) > pn for some parameter p, it is 
not hard to verify that with constant probability a{El) will be at least ps. This 
follows from the fact that the expected size of iL n I is at least ps. However, if 
a{G) < pn, and this is our only assumption on G, we cannot hope for a positive 
bound on the probability that a{H) < ps (unless s is equal to n). 

Thus, in the latter case, we strengthen our assumption on G to hypergraphs 
G which not only satisfy a{G) < pn but are also far from having an independent 

* Part of this work was done while visiting the School of Computer Science, Tel Aviv 
University, Tel Aviv 69978, Israel. 
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set of size fm (we defer defining the exact notion of “far” until later in this dis- 
cussion). That is, given a hypergraph G which is far from having an independent 
set of size pn, we study the minimal value of s for which (with high probability) 
a random subgraph of size s does not have an independent set of size ps. This 
question (and many other closely related ones) have been studied in (2-uniform 
hyper) graphs in [GGR98] under the title of property testing. 

Property Testing: Let C be a class of objects, and V a property of objects from 
C. Property testing addresses the problem of distinguishing between elements 
c £ C which have the property V and elements that are far from having the 
property V. The aim is to construct efficient (randomized) distinguishing algo- 
rithms that sample the given element c in relatively few places. The notion of 
property testing was first presented by Rubinfeld and Sudan in [RS96] where 
the testing of algebraic properties of functions was addressed. Goldreich, Gold- 
wasser, and Ron [GGR98] later initiated the study of combinatorial objects in 
the context of property testing. In their work they studied (2-uniform) graphs 
and considered several fundamental combinatorial graph properties related to 
the independence number, chromatic number, size of maximum cut, and size of 
the maximum bisection of these graphs. Since, many papers have addressed the 
notion of property testing, both in the context of functions and in the combina- 
torial setting {e.g. see surveys [Gol98,Ron01,Fis01]). 

Property testing of hypergraphs has also been studied in the past. Gzu- 
maj and Sohler [GSOl] initiated this line of study when analyzing the prop- 
erty of being £ colorable. Golorability, and other properties of /c-uniform hy- 
pergraphs (that can be phrased as a Max-fc-GNF formula) were also studied in 
[AdlVKK02,AS03]. In this work, we consider testing the independence number of 
hypergraphs. To the best of our knowledge, this property has not been addressed 
in the past (in the context of hyper graphs). 

Testing the Independence Number: Goldreich, Goldwasser, and Ron [GGR98] 
study property testing of the independence number of (2-uniform) graphs. In 
[GGR98] a graph G of size n is said to be e-far from having an independent set 
of size pn if any set of size pn in G has at least induced edges. It was shown 
in [GGR98] that if G is £-far from having an independent set of size pn then 
with high probability a random subgraph of size s = £I2 SlLUi1p^ for a sufficiently 
large constant c, does not have an independent set of size ps. The sample size 
s was later improved in [FLS02] to c^log(-) (again c is a sufficiently large 
constant). It is not hard to verify that this implies a (two-sided error) property 
testing algorithm for the independence number of G. Namely, given a graph G, 
one may sample a random subgraph iJ of G of size s, and exhaustively compute 
a{H). On one hand, if G happens to have an independent set of size pn, then 
with some constant probability pi the independence number of H will be at least 
ps (recall that the expected value of a{H) is at least ps). On the other hand, if 
G is £-far from having an independent set of size pn then, as mentioned above, 
with at most some small probability p 2 (which is set to be smaller than p\ ) it is 
the case that a{H) > ps. 
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Our Results: In this work, we address the study of hypergraphs in the above 
context. Namely, given a fc-uniform hypergraph G = (V, E) of size n, we show 
that if one must remove at least en^ edges of G in order for the remaining 
hyper graph to have an independent set of size pn (i.e. G is e-far from having an 

2fc 

independent set of size pn) then a random subgraph i/ of G of size s ~ c(k)t^ 
satisfies a{H) < ps with high probability. Here c{k) depends on k alone, which 
implies that the sample size s is independent of n. We also show a lower bound 

2 fe — 1 

for the size of s of value c'{k) ^ a (where c'(fc) is yet an additional function 
which depends on k alone). 

Definition 1 Let A he a subset ofV. Define E(A) to be the set of edges in the 
hypergraph induced by A. 

Definition 2 Let p < 1. A k-uniform hypergraph G = (V,E) is said to be 
{p,e)~ connected iff every subset A of V of size pn satisfies |if(^)| > (i.e. the 

number of edges in the subgraph induced by A is greater than en^ ). 

Theorem 3. Let G he a k-uniform hypergraph. Let LI he a random sample of 

, 2k , , 

G of size s > c2^k\^ log for a large constant c. 

1. If G has an independent set of size pn, then with probability > 1/4 the 
subgraph H will have an independent set of size ps. 

2. IfG is {p, e) -connected then with probability < 1 /20 the subgraph H will have 
an independent set of size ps. 

A few remarks are in place. First of all, in the above theorem we consider only 
hypergraphs which are fc-uniform. Theorem 3 can be extended to hypergraphs 
in which any edge is of size at most k. This follows from the fact that any 
sufficiently large hypergraph which is {p, e)-connected with edges of size k and 
smaller contains a subgraph (on the same vertex set) which is fc-uniform and 
(p, e/2)-connected. Secondly, throughout this work we analyze the properties of 
random subsets H which are assumed to be small. Namely, we assume that the 
value of s (as stated in Theorem 3) and the parameters p, e and n satisfy (a) 
s < Cy/n and (b) s < cpn for a sufficiently small constant c. These assumptions 
typically hold in the context of property testing. Finally, the constants 1/4, and 
1 /20 presented above are not tight (we have not made an attempt to find the 
tightest possible constants). 

Proof Techniques: Given a fc-uniform hypergraph G with vertex size n which 
is (p, £)-connected, the bulk of our work addresses the study of the minimal 
value of s for which a random subgraph H oi G satisfies a{H) < ps with high 
probability. This is done by analyzing the probability that a random subset H of 
G of size s satisfies a{H) < ps (as a function of s, p and e). Our proof structure 
is strongly based on that appearing in (the journal version) of [FLS02], and 
proceeds as follows. Given a sample size s, we start by bounding the probability 
that a random subset i? of G of size £ > ps is an independent set. As any subset 
Roi H is random in G, one may bound the probability that oi{H) > ps using the 
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standard union bound on all subsets i? of i? of size > ps. We analyze this naive 
strategy and show that it only suffices to bound the probability of a slightly 
stronger condition than the condition a{H) > ps. Namely, using this scheme, we 
bound the probability for which a{H) > 3ps (instead of exactly ps). The naive 
strategy is then enhanced in order to bound the probability that a{H) > Sps as 
a function of <5 for any <5 > 1. This now suffices to bound the probability that 
a{H) > ps (using a few additional ideas). 

Applying the proof technique of [FLS02] to the case of /c-uniform hypergraphs 
involves several difficulties. Roughly speaking, these are overcome by considering 
for each vertex v in the given graph G, a weighted set system of all subsets of V 
which share an edge with v. This set system plays the role of the standard “set 
of neighbors” considered when studying (two-uniform) graphs. Specifically, for 
all vertices v we consider the set system Ay consisting of sets a = {ui , . . . , Ui} 
for which there exists an edge e in G which includes both the vertex v and the 
set a. The weight applied to each subset is fixed to be a function of the subset 
size. Ideas along this line have been used in the past in the study of hypergraphs 
{e.g. [AS03]). 

Organization: The remainder of the paper is organized as follows. In Section 2 
we present the proof of Theorem 3. In Section 3 we present our lower bounds on 
the sample size s. As mentioned above, our proof uses many of the arguments 
presented in [FLS02]. During our presentation, we repeat quite a few of these 
arguments, in aim to make the paper comprehensible. Due to space limitations, 
some of our results appear without detailed proof. Most of the details can be 
found in an extended version of this paper [Lan03] . 



2 Proof of Theorem 3 

In this section, we prove Theorem 3. The proof of Theorem 3 (I) is straight- 
forward and (due to space limitations) is omitted. The proof of Theorem 3 (2) 
follows. 



2.1 The Naive Scheme 

Let G = {V, E) be a (p, e)-connected fc-uniform hypergraph. In this section we 
study the probability that a random subset R of V of size £ is an independent 
set. We then use this result to bound the probability that a random subset H 
of G of size s has a large independent set. 

We would like to bound (from above) the probability that R induces an 
independent set. Let {ri,...,r^} be the vertices of R. Consider choosing the 
vertices of R one by one, such that at each step the random subset chosen so 
far is Ri = {ri,...,ri} and the vertex r^+i is chosen from V \ Ri. Assume 
that at some stage Ri is an independent set. We would like to show (with high 
probability) that after adding the remaining vertices {ri+i, . . . , r^} to Ri, the 
final set R will not be an independent set. 
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The vertices in V \ Ri that cannot be added to Ri are exactly the vertices 
V that share an edge with some k — 1 vertices in Ri (namely, these vertices will 
result in a set Ri+i that is no longer independent). Let N{Ri) = Ni be the set of 
such vertices in V\Ri, and let I{Ri) = R be V\Ni. Consider the next random 
vertex rj+i € R. If rj+i is chosen from Ni then it cannot be added to Ri, and we 
view this round as a success regarding the set Ri. Otherwise, Vi+i happens to be 
in li and can be added to Ri. But if the addition of to Ri happens to add 
many vertices to Ni, we also view this round as a successful round regarding Ri. 

Motivated by the discussion above, we continue with the following definitions. 
For each subset Ri we define the following weighted set systems of subsets of V. 
Let RES{Ri, 1) (for restrict) be the set of singletons {u} that share an edge with 
vertices in Ri. Namely, {u} G RES{Ri, 1) iff there exists vertices {wi, . . . , Wk-i} 
in Ri such that there is an edge {f, wi, . . . ,Wk-i} in E. RES{Ri, 1) is exactly 
the set Ni defined above. Define the weight of each element in RES{Ri,l) as 
n^~^. Let RES{Ri,2) be the set of pairs of vertices vi, V 2 which (together) 
share an edge with vertices in Ri. Namely, {^ 1 ,^ 2 } G RES{Ri, 2) iff there exists 
vertices {wi, . . . , Wk- 2 } in Ri such that there is an edge {vi,V 2 , wi, . . . , Wk- 2 } 
in E. Define the weight of each element in RES{Ri,2) as Similarly for 

each j G {1, . . . , fc — 1} let RES{Ri,j) be the set of subsets {v\, . . . , vj} of V 
of size j that share an edge with vertices in Ri. Define the weight of each ele- 
ment in RES{Ri, j) as Finally let RESi = RES{Ri) be the union of 

the sets RES{Ri,j) where j G {!,..., fc — 1}. Let \\RES{Ri, j)\\ {\\RESi\\) de- 
note the weight of elements in RES{Ri, j) {RESi). Notice that \\RES{Ri, j)\\ < 

Definition 4 Let the normalized degree w.r.t. Ri of a vertex v € V be the 
amount on which v restricts upon Ri: 

d„{R,) = \\RES{R, U M)|| - \\RES{Ri)\\ = \\RES{R^ U W) \ RES{R^)\\. 

In the above notice that RES{Ri) C RES{Ri U {u}). We call a vertex v in V 
heavy with respect to Ri (or Ri-heavy for short) if dv{Ri) > 2 k(k- 3 )\ 

Each subset Ri of V now defines the following partition {LR, Hli, Ni) of V 
and the set RESi. Let RESi and Ni be as defined as above. Let R = V \ Ni. 
R is now partitioned into two parts: vertices in R with low normalized degree 
(w.r.t. Ri), denoted as the set LR, and vertices with high normalized degree, 
denoted as Hli. Namely LR is defined to be the pn vertices of li with minimal 
normalized degree and Hli is defined to be the remaining vertices of li. Ties are 
broken arbitrarily or in favor of vertices in Ri (namely, vertices in Ri are placed 
in Lli before other vertices of identical degree). If it is the case that \Ii\ < pn 
then Lli is defined to be li, and Hli is defined to be empty. 

We define the partition corresponding to Rq = <j) as {LIq, HIq, No), where 
LIq are the pn vertices of G of minimal normalized degree, HIq are the remaining 
vertices of G, and No is empty. RESq is also defined to be empty. 

Notice, using this notation, that the subset Ri is an independent set iff Ri C 
R. Moreover, in this case Ri C Lli (all vertices of Ri have normalized degree 0). 
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Furthermore, each vertex n in an independent set R = Ri = {r\, . . . ,n} satisfies 

€ R—1 ■ 

We are now ready to bound the probability that a random subset R = 
{ri, . . . , rg} of G is independent. Let Ri = {ri, . . . , r^}, and let {LR, HR, Ni) 
be the corresponding partition of V defined by Ri. Consider the case in which 
R is an independent set. As mentioned above, this happens iff for every i the 
vertex is chosen to be in = LR-i U HR-i. We would like to show that 
this happens with small probability (if i is large enough) . 

Consider the set RESi as we proceed in the choice of vertices in R. Initially, 
the subset RE So is of weight 0, and it gets larger and larger as we proceed in 
the choice of vertices in R. Each vertex in ri e HR-i increases the weight of 
RESi-i substantially, while each vertex in LR-i may only slightly change the 
weight of RESi-i- In the following, we show that there cannot be many vertices 
Ti G R that happen to fall into HR-i. We thus turn to consider vertices Vi that 
fall in LR-i (there are almost £ such vertices). The size of LR is bounded by 
pn. Hence, the probability that S LR is bounded by p (by our definitions 
Ri-i C LR-i and the vertex ri is random in C \ i?i_i). This implies that the 
probability that R is an independent set is roughly bounded by p^ . Details follow. 

Lemma 1. Let G he a (p, e) -connected hypergraph. Let R = {ri, . . . , r/} he a set 
in G. The number of vertices ri which satisfy ri G HR-i is hounded hy t = ^ . 

Proof. We start with the following claim which is the main technical contribution 
of this work. 

Claim 5 Let Ri be as defined above, and let (LR, HR, Ni) and RESi he its cor- 
responding partition and set system. Let R = LR UHR. If G is (p,e)- connected 
then, every vertex in HR is Ri-heavy. 

Proof. Assume that Lli = pn (otherwise Hli is empty and the claim holds). By 
our assumptions on G we have that LR induces at least en^ edges. Let E{LR) be 
the set of these edges, and let m be the size of E{LR). To simplify our notation, 
let LI = LR, HI = Hli, N = Ni and R = Ri. Now that the index i is free, we 
use it in the following new context. For z€{l,...,fc— 1} consider the following 
sequence of weighted z-uniform hypergraphs Hi = {LI, Ei) which all have vertex 
set LI. The edge set Ei is defined to be all subsets of size z of edges in E{LI) 
(here and throughout our work we consider a (hyper) edge of size k as & subset 
of vertices of size k). For example, each edge in E{LI) induces (2) edges in H2, 
(3) edges in H^, and so on. The weight of an edge in Hi is equal to the number 
of edges e in E{LI) which satisfy Cj C e. 

Recall the sets RES{R,j) and RES{R). Our goal is to prove the existence 
of a vertex v G LI which is i?-heavy (this will imply our assertion). Namely a 
vertex v which when added to R will significantly increase the weight of RES{R). 
Formally we are looking for a vertex v G LI which satisfies \\RES{RU {u})|| — 
||i?i?S'(i?)|j > 2fc(fc-3)! f Notice that the weight of a subset of vertices with 
respect to RES{R) may (and usually does) differ from its weight with respect 
to Hi. To avoid confusion we denote the weight of a subset e with respect to 
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RES{R) as WR{e). Recall that every (partial) edge e of size i in RES{R) has 
weight tUfi(e) = The weight of an edge e in Eli will be marked as WHi (e). 

For each i we now consider the weight (with respect to Hi) of the edges in 
Hi which are in RES{R). Denote this weight as resi. Notice that resi = 0 (by 
our construction LI n N = (j), which implies that singletons {f } C LI are not in 
RES{R)). We consider the following cases. 

Case 1: We start by assuming that resk-i < {k — l)m. As the total weight of 
edges in Hk-i is km, we conclude that the weight of edges in Hk-i that are not 
in RES{R) is at least m. Let w be a vertex in LI. We say that an edge e in 
Hk-i is a w-edge if there exists an edge in E{LI) consisting of the union of e and 
{?;}. Notice that an edge e in Hk-i of weight WHk_i{e) is actually a v edge for 
w^Hfe„i(e) distinct vertices v. As each edge in Hk-i is a w-edge for some vertex 
V in LI, we have the existence of a vertex v in LI with at least ^ (distinct) 
w-edges that are not in RES{R). The weight of each such w-edge in RES{R) is 
1. Furthermore, each such w-edge appears in RES{RU {w}) implying that 

||RAS'(RU {v})|| - ||i?A5'(i?)|| > — > 

pn p 

which in turn implies the v is i?-heavy. 

Cases 2 to k — 1: Starting at z = fc — 1 and iteratively continuing until z = 2 
consider the following cases. From the previous step, we may assume that reSi > 
(fcZjy ■ We now also assume that resi-i < ■ Consider an edge 

e = (vi,..., Vi) in Hi of weight WHi (e) which contributes to resi. Each such edge 
induces z edges in Hi_i: {e), . . . , e'} (each edge obtained by removing one vertex 
from e). We are interested in bounding (by below) the weight of edges e as above 
with corresponding edges e' which are not in RES{R) (here j G {l,...,z}). 

By our construction, the weight of any edge e' in iLi_i equals the number of 
edges in G that include e'. It is not hard to verify that this equals times 

the weight of edges in Hi which include e' . We conclude that the weight of edges 
e in Hi which are in RES{R) with some corresponding edge e' in Hi-i which is 
also in RES{R) is bounded by resi-i{k — z + 1). This leaves us with 

-IN m (k — 1) 

res, -{k-i+ l)res,_i > 

edges e which contribute to reSi for which e' for j G {l,...,z} are not in 
RES{R). 

We now conclude the existence of a vertex v G LI which is adjacent to (z.e. 
is included in) at least the weight of {k-i)\ ®dges in Hi which appear in 

RES{R) such that their corresponding edges in Hi_i are not in RES{R). The 
weight of each edge in Hi is bounded by < (pn)^“*. Thus there are at least 

2 >=-'(pn)’^-'+^ (k-i)< distinct edges in Hi adjacent to v with corresponding edges 
in Hi-i that are not in RES{R). 
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We will now show that v is i?-heavy. Consider one of the distinct edges 
e = (v,vi,. . . , Vi-i) as discussed above. By our assumption the set {z;i, . . . , Vi-i} 
is not in RES{R). The edge e appears in RES{R) implying that there exists 
vertices {wi, . • . , Wk-i} (in R) such that {v,vi, . . . , Vi-i,wi, . . . , Wk-i) is an edge 
in the original hypergraph G. This in turn implies that {ui, . . . will be 

included in RES{R U {f}). As each set of size t — 1 in RES{R) has weight 
Wfi = and there are at least (k-i)\ distinct edges of interest 

adjacent to v, we conclude that 

||i?A5(i?U M)|| - ||i?A5(i?)|| > 

Now to prove our lemma, consider the subsets Ri = {ri, . . ,,ri\ and their 
corresponding partitions {LR, Hli, Ni). Let R = LR U HR. We would like to 
bound the number of vertices rt that are in HR-i. Consider a vertex rt in HR. 
By Claim 5, its normalized degree w.r.t. Ri is at least 2 i^(k- 3 y. ■ RES{(j)) 

is initially empty, and for any i, RES{Ri) is of weight at most J2i=2 — 

Each vertex Ti G HR-i increases by at least 2 fc(fc- 3 )! ■ 

We conclude that there are at most vertices ri in R which are in HR-i. 

Let (5 be a large constant. “Plugging” Lemma 1 into the proof appearing in 
(the journal version of) [FLS02], we are now able to bound the probability that 
a random subset iL of G of size s has an independent set of size > Sps. This is 
done in two steps, first we bound the probability that a random subset i? of G 
is independent. Then, using the union bound, we obtain the result mentioned 
for H. In the upcoming Section 2.2 we present the refined proof technique of 
[FLS02] (which gets rid of the parameter d) applied to the case of hypergraphs. 
Namely, we will bound the probability that a random subset iL of G of size s 
has an independent set of size > ps. 

Lemma 2 ([FLS02]). Let G he a {p^e) -connected hypergraph. Let t he as in 
Lemma 1. Let I > 2t. The prohahility that i random vertices of G induce an 
independent set is at most 




Corollary 6 ([FLS02]) Let G he a {p,e)~ connected hypergraph. Let t he as in 
Lemma 1. Let H he a random sample of G of size s. Let S > e, and let c he a 
sufficiently large constant. If s > thgfi thg prohahility that a{H) > dps 

is at most 

2.2 An Enhanced Analysis 

Recall the proof technique from Section 2.1. We started by analyzing the prob- 
ability that a subset i? of of size £ is an independent set. Afterwards we 
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bounded the probability that a{H) > 6pn (using the standard union bound on 
all subsets i? of of size greater than i = 6pn). In this section we enhance 
the first part of this scheme by analyzing the probability that a subset R of H 
of size £ is a maximum independent set in H (rather than just an independent 
set of H). Then, as before, using the standard union bound on all large subsets 
R of H, we bound the probability that a{H) > ps. We show that taking the 
maximality property of R into account will suffice to prove Theorem 3 (2). 

Let H = {hi , . . . , hs} be s random vertices in G. We would like to analyze 
the probability that a given subset i? of of size ^ is a maximum independent 
set. Recall (Section 2.1), that the probability that R is an independent set is 
bounded by approximately p^ . An independent set i? is a maximum independent 
set in H only if adding any other vertex in H to R will yield a set which is no 
longer independent. Let R= Ri he an independent set, and let {LIi,HIe,Ne) 
be the partition (as defined in Section 2.1) corresponding to R. Consider an 
additional random vertex h from H . The probability that i? U is no longer 
an independent set is approximately |iV^|/n (here we assume that |i?| is small 
compared to n). The probability that for every h G H \ R the subset i? U {ft-} 
is no longer independent is thus ~ (|i\ft|/n)® Hence, the probability that a 
given subset R of H of size £ is a maximum independent set is bounded by 
approximately p^ (|iV^|/n)*~^. This value is substantially smaller than p^ iff |7V^| 
is substantially smaller than n. We conclude that it is in our favor to somehow 
ensure that |A^^| is not too large. We do this in an artificial manner. 

Let R = jri, . . . , ri\ be an independent set, let Ri = jri, . . . , r^j, and let 
{LIi,HIi,Ni) be the partition (as defined in Section 2.1) corresponding to Ri. 
Let RE Si be the set system corresponding to Ri. Roughly speaking, in Sec- 
tion 2.1, every time a vertex was chosen, the set system RE Si and iVj was 
updated. If was chosen in then RESi-i increased substantially and 

W-i potentially also grew substantially, and if was chosen in both 

RESi-i and Aft_i only slightly changed. We would like to change the definition 
of the partition {Lli, Hli, Ni) and of RE Si corresponding to Ri as to ensure 
that Ni is always substantially smaller than n. This cannot be done unless we 
relax the definition of Aft. Recall that Aft was defined (in Section 2.1) to be the 
set of vertices v which share an edge with vertices in Ri. Specifically, Aft is the 
set of vertices v for which there exists vertices (wi, . . . ,Wk-i} in Ri such that 
there is an edge {v,wi, . . . ,Wk-i} in E. In this section Aft will only include a 
subset of these vertices (a subset which is substantially smaller than n). Namely, 
in our new definitions RESi-i and Aft_i will be changed only if was chosen 
in HIi_i. In the case in which G LA-i U Aft_i, we do not change Aft_i at all. 
As we will see, such a definition will imply that |Aft| < (1 — p)s, which will now 
suffice for our proof. 

A New Partition and Set System: Let El = (fti, . . . , hs} be a subset of V. Let 
Ri = jri, ... , n} be a subset of H of size i. In the previous section, the subset 
Ri defined a partition {Lli, HR, Ni) of V and a set system RESi. In this section, 
for each i we will define a subset Ri of Ri, and a new partition and set system. 
The new partition and set system corresponding to Ri will be defined similarly 
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to those defined in the previous section with the exception that Ri will play the 
role that Ri played previously. The set Ri, the new partition {LR, HR, Ni), and 
the set system RE Si are defined as follows (as before, let R = LR U HR). 

1. Initially Rq = Rq = <f>, LR is the pn vertices in V of minimal normalized 
degree w.r.t. Ri, HR = F \ LR, and Nq = (j). In the above, ties are broken 
by an assumed ordering on the vertices in V. RE So = RES{Rq) is defined 
to be empty. 

2. Let Ri, (LR,HIi,Ni) and RESi be the sets corresponding to Ri, let 

be a new random vertex. Let Ri+i = RiU {ri+i}, we now define the sets 
Ri+i, (LR+i,HIi+i,Ni+i) and RESi+i. Let N{n+i) be the set of vertices 
which share an edge with vertices in Ri U Tj+i. We consider the following 
cases: (a) If G LR or n+i G Ni then the sets corresponding to Ri+i 
will be exactly those corresponding to Ri. Namely, Ri+i = Ri, LR+i = LR, 
HR+i = HR, W+i = Ni, and RESi+i will be defined as RESi. (b) If 
Ti+i G Hli then we consider two sub-cases: (bl) If |WUfV(ri+i)| < {1 — p)n, 
then Ri+i = RtU {rj+i}, RES^+i = RES{R^+l), and LR+i, HR+i, fV^+i, 
are defined as in Section 2.1. Namely, W+i = NiU N{ri+i). R+i is defined 
to be \ fVi+i. LR+i is defined to be the pn vertices of with minimal 
normalized degree w.r.t. Ri+\. Finally, HIij,.i is defined to be the remaining 
vertices of Ii+\. Ties are broken by the assumed ordering on V. (b2) If 
\Ni U N{n+i)\ > (I — p)n, then let N{n+i) be the first (according to the 
assumed ordering on V) (1 — p)n — |A^i| vertices in N{ri+i) and set W+i = 
Ni U iV(ri+i). Furthermore, set LR^i to be the remaining pn vertices of 
G, and HR to be empty. Notice that in this case |iVi+i| is of size exactly 
(1 — p)n. Finally, let i?i+i = Ri and RESi-\.\ = RESi. 

A few remarks are in place. First of all it is not hard to verify that the 
definition above implies \Ni\ < {l — p)n for all i G Secondly, due to the 

iterative definition of our new partition, the sets corresponding to the subsets 
Ri depend strongly on the specific ordering of the vertices in Ri. Namely, in 
contrast to the partitions (set systems) used in Section 2.1, a single subset R 
with two different orderings may yield two different partitions (set systems). For 
this reason, in the remainder of this section, we will assume that the vertices of 
H are chosen one by one. This will imply an ordering on H and on any subset 
R of H. The partitions we will study will correspond to these orderings only. 

Finally, in Section 2.1, an (ordered) subset R = {ri, . . . , ri} was independent 
iff Vi ri G R-i (according to the definition of li-i appearing in Section 2.1). In 
this section, if R is independent then it still holds that Vi G R-i. However, 
it may be the case that Vi G R-i but R is not an independent set. In the 
remainder of this section, we call ordered subsets R for which Vi r* G li-i free 
sets. We analyze the probability that a random ordered subset H oiV of size s 
does not have any free sets R of size larger then ps. This implies, that H does 
not include any independent sets of size ps. 

We now turn to prove Theorem 3 (2). Roughly speaking, we start by analyz- 
ing the probability that a random subset i? is a free set. We then analyze the 
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probability that a given subset i? in i/ is a maximum free set. Finally, we use 
the union bound on all subsets R of H of size > ps to obtain our results. 

We start by stating the following lemma which is analogous to Lemma 1 from 
Section 2.1. The main difference between the lemma below (and its proof), and 
that of the previous section is in the definition of the partition {LR, Hli, Ni), 
and the set system RE Si. Proof of the lemma is omitted. 

Lemma 3. Let G he a {p, e)- connected hypergraph. Let R = {ri, . . . ,ri} be an 
ordered set in G of size £. The number of vertices rt which satisfy ri G iLL-i is 
bounded by t = ^ . 

We now address the probability that a random subset i? of iL is a maximum 
free set. We then use the union bound on all subsets i? of iL of size > ps (and 
a few additional ideas) to obtain our main result. Due to space limitations, the 
proof of the following Lemma and Theorem is omitted. The interested reader 
can find their proofs in [Lan03] (or proofs of similar nature in [FLS02]). 

Lemma 4. Let G be a {p^e) -connected hypergraph. Let t be as in Lemma 3. Let 
£ > 2t. Let H be an ordered random sample of G of size s > £. The probability 
that a given subset R of H of size £ is a maximum free set is at most 

Theorem 3 (2). Let G he a {p, e)-connected hypergraph. Let t be as in Lemma 3. 
Let H he a random sample of G of size s. Let c he a sufficiently large constant. 

2fc-l , . 

If s > ct ' ^‘2 log(f) then the probability that H has an independent set of size 
> ps is at most 

3 Lower Bounds on the Sample Size s 

Roughly speaking. Theorem 3 (2) states that given a {p, e)-connected hypergraph 

2k 

G, a random sample iJ of G of size s proportional to 2^k\^ (or larger) will not 
have an independent set of size ps (with high probability). We now continue to 
study the minimal value of s for which a{H) < ps with high probability, and 

2fc-l 

present a lower bound of ^^ 2^2 on the size of s. 

Lemma 5. Let n he a sufficiently large constant. Let p > 0 and £ > 0 satisfy 

k 

(a) e « (b) « n and (c) « pn (here a « b if for a 

sufficiently small constant 5 > Q it holds that a < 5b). There exists a graph G on 
n vertices for which G is {p,s) -connected, and with probability > 1/20 a random 

2fc-l 

subgraph H of size s = ^^ 2^2 will have an independent set of size ps. 

Proof. Consider the /c-uniform hypergraph G = (V,E) in which (a) \V\ = n, 

(b) V consists of two disjoint sets A and P \ T, where A is of size (1 — ^^)pn, 
and (c) the edge set E of G consists of all subsets of V of size k except those 
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included in A (namely, A is an independent set). On one hand, every subset 
of size pn in G induces a subgraph with at least edges (implying that G 
is (p, £)-connected) . On the other, let H he & random subset of V obtained by 
picking each vertex independently with probability - ■ The expected size 

2fe-l 

of is 5 = ■ In following, we assume H is exactly of size s, minor 

modifications in the proof are needed if this assumption is not made. The set 
i/ n ^ is an independent set in the subgraph induced by H. The expected size of 
H n Ais (1 — ^^)ps. Let N{0, 1) denote a standard normal variable. It can be 
seen using the central limit theorem (for example [Fel66]) that for our choice of 
parameters, the probability that \H H A\ is greater than ps + y/JJs is 

at least Pr [A^(0, 1) > 3/2] > In such a case the size of H C\ A will be greater 
than (1 — ^^)ps + yfps = ps for our value of s. Hence implying the lemma. 
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Abstract. We describe a deterministic algorithm that, for constant k, 
given a fc-DNF or fc-CNF formula ip and a parameter e, runs in time linear 
in the size of ip and polynomial in 1 je (but doubly exponential in k) and 
returns an estimate of the fraction of satisfying assignments for p up to 
an additive error e. This improves over previous polynomial (but super- 
linear) time algorithms. The algorithm uses a simple recursive procedure 
and it is not based on derandomization techniques. It is similar to an 
algorithm by Flirsch for the related problem of solving fc-SAT under the 
promise that an e-fraction of the assignments are satisfying. Our analysis 
is different from (and somewhat simpler than) Hirsch’s. 

We also note that the argument that we use in the analysis of the algo- 
rithm gives a proof of a result of Luby and Velickovic that every fc-CNF 
is “fooled” by every 5-biased distribution, with 5 — 1/2°*'*^^ \ 



1 Introduction 

We consider the following problem: given a fc-CNF formula p and a parameter 
e, approximate within an additive error e the fraction of satisfying assignments 
for tp 

The problem is easy to solve using randomization: just generate 0(l/e^) 
assignments at random and then output the fraction of assignments in the sample 
that satisfies ip, and the question is whether efficient deterministic algorithms 
exist. 

We also consider the related problem of finding a satisfying assignment for 
p under the promise that an e fraction of assignments are satisfying. Again, 
we are interested in deterministic algorithms, and the problem is easy to solve 
probabilistically, since after picking 0{l/e) assignments at random it is likely 
that one of them satisfies the formula. 

One can consider the approximate counting problem as the problem of de- 
randomizing two-sided error algorithms implemented by depth-two circuits. The 

* Supported by NSF grant CCR-9984703, a Sloan Research Fellowship and an Okawa 
Foundation Grant. 

^ Note that an algorithm achieving additive approximation e for fc-CNF immediately 
implies an algorithm achieving the same additive approximation for fc-DNF. Also, 
achieving multiplicative approximation (1 -f e) for fc-DNF reduces to achieving ad- 
ditive approximation since a satisfiable fc-DNF is satisfied by at least a 1/2^ 

fraction of assignments. 
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problem of finding a satisfying assignment for tp under the promise that that 
there is a large number of such assignments can be seen as the problem of de- 
randomizing one-sided error algorithms implemented by depth-two circuits. 

These problems were first studied by Ajtai and Wigderson [AW89]. Using 
derandomization techniques (specifically, t-wise independence) they give an al- 
gorithm for the counting problem running in time 0{n^ -\- ) and an 

algorithm for the satisfiability problem running in time 0{n^^ i°g(i/e)). They 
also give sub-exponential time algorithm for the counting problem for functions 
computed by AC° circuits^. 

The algorithm of Ajtai and Wigderson for fc-CNF could be improved by using 
almost t-wise independent distributions, for example the small bias distributions 
of [NN93], instead of distributions that are perfectly t-wise independent. For 
constant e, this would improve the running time to roughly n • (logn)*^^^^ • 

2^ ^ ' for both the approximate counting problem and the satisfiability problem. 
Almost t-wise independent distributions were introduced after the publication 
of [AW89]. 

Nisan [Nis91] and Nisan and Wigderson [NW94] construct a pseudorandom 
generator that fools constant-depth circuits and that has poly-logarithmic seed 
length. As a consequence, they achieve ' time algorithms for the count- 

ing and satisfiability problems for AC° circuits. 

Luby, Velickovic and Wigderson [LVW93] optimize the constructions of Nisan 
and Wigderson [Nis91,NW94] to the case of depth-2 circuits, thus solving the 
counting and satisfiability problem in time ^ for general CNF and DNF. 

Luby and Velickovic [LV96] show how to reduce an arbitrary CNF and DNF to 
a formula in a simplified format, and show that the counting and satisfiability 
problems can be solved in polynomial time for fc-CNF even if fc = 0((log is 

more than a constant. Luby and Velickovic [LV96] also also present an improved 
derandomization of general CNF and DNF that runs in slightly super-polynomial 

time Finally, Luby and Velickovic also show that every /c-CNF is 

“fooled” by every distribution that is close to being t(fc)-wise independent, where 
t{k) = \ (A distribution X “fools” a function /() if the probability that 

f{X) = 1 is approximately the same as the probability that /() equals 1 under 
the uniform distribution.) 

Hirsch [Hir98] shows how to solve the satisfiability problem for fc-CNF in time 
0(Lfc(2/e)®(^)), where L <nk \s the size of the formula and B{k) is a function 
for which a closed formula is not given, but that seems to grow exponentially in 
fc. Hirsch’s algorithm does not use derandomization techniques. 

In this paper, we show how to solve the approximate counting problem and 
the satisfiability problem in time 0{L {\ / ). 

Our algorithm is based on the following simple observation: given a fc-CNF 
ip, then for every fixed c, either we can efficiently find a set of < fcc variables 

^ An AC^ circuit is a circuit of constant depth and polynomial size with unbounded 
fan-in AND and OR gates. A CNF formula is a depth-two AC° circuit, and so is a 
DNF formula. 
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that hits all the clauses, or we can efficiently find > c clauses over disjoint sets 
of variables. In the former case, we can try all assignments to those variables, 
and recurse on each assignment, thus reducing our problem to 2^'^ problems on 
{k — 1)-CNF instances; in the latter case, less than a (1 — 1/2^)'^ fraction of 
assignments can satisfy <p, and thus 0 is an approximation within an additive 
error (1 — 1/2^)'^ of the fraction of satisfying assignments for (p. Fixing c to be 
2^ In 1/e gives us the main result. 

Using the same recursive approach adopted in our algorithm, we give a 
slightly different proof of the result of Luby and Velickovic about fc-CNF being 
fooled by almost t-wise independen distributions. We show that every k-CNF 
is well approximated, in a certain technical sense, by a decision tree of depth 
t = 0{k2^), and it is well known that functions that are well approximated (in 
the above technical sense) by a decision tree of depth t cannot distinguish the 
uniform distribution from a distribution that is approximately t-wise indepen- 
dent. 



2 The Recursive Algorithm 

We describe the algorithm only for the case of fc-CNF. As discussed in the 
introduction, an algorithm for fc-DNF is an immediate corollary of the algorithm 
for fc-CNF. 

We use the following simple fact. 

Lemma 1. There is an algorithm that, on input a k-CNF formula p and a 
parameter t, runs in time linear in the size of p and then it returns either a set 
oft clauses over disjoint sets of variables, or a set S of at most k{t— 1) variables 
such that every clause in the formula contains at least one variable from S. 

Proof. (Sketch) Consider the fc-uniform hypergraph H that has a vertex for 
every variable and an hyperedge for every clause. It is easy to find a maximal 
matching in H in linear time, that is, a set S of clauses over disjoint sets of 
variables and such that every other clause in p shares some variables with some 
clause in S. If [S'! > t, then we return t of the clauses in S. Otherwise, we return 
the set of < A: • [S'! < k{t — 1) variables that occur in the clauses of S. Such a set 
of variables clearly “hits” all the clauses of p. □ 

The algorithm works as follows: given p and e, 

— If 1 ^ is a 1-CNF, that is, it is just an AND of literals, then we output 0 if 
there are two inconsistent literals and 2“'^ where c is the number of distinct 
literals, otherwise. This procedure is exact and can be implemented in linear 
time. 

— Otherwise, we let t be the smallest integer such that (1 — 1/2^)*+^ < e, 
so that t < 2^(lnl/£), and we run the algorithm of Lemma \ on p with 
parameter t -|- 1. 
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• If the algorithm of Lemma 1 finds t+1 clauses Ci , . . . , Ct+i over disjoint 
sets of variables, then it is clear that the probability that (p is satisfied 
by random assignment is at most e, and we return the value 0 as our 
approximation . 

• If the algorithm of Lemma 1 finds a set V of at most tk < /c2^1n(l/e) 
variables that hit all the clauses, then, for every assignment a to the 
variables V, define tpa to be the formula obtained from p by substituting 
the assigment into the variables. Note that Pa is a (/c — 1)-CNF formula. 
We recurse on each of the pa with parameter e, and take the average 
of the results. Assuming that each recursive call returns an £ additive 
approximation, the algorithm returns an e additive approximation. 

If we denote by T{L,k) the running time of the algorithm for a k-CNF 
instance of size L, then we have 



T{L,1) = 0{L) 



and 

T(L, k) < 0{L) + k-1) 

which solves to T{L,k) = 0{L ■ 22 fe(im/e) 2 '“) ^ 0(L(l/£)(‘''4)fe2'“)^ 

For the promise problem of finding a satisfying assignment under the promise 
that an e fraction of assignments are satisfiable, we essentially use the same re- 
cursive algorithm. When we are down to 1-CNF, we find a satisfying assignment 
or fail if the instance is unsatisfiable. (Indeed, we can stop at 2-CNF.) In the 
recursive step, we fail if t is such that (1 — 1/2^)* < e. The analysis of the run- 
ning time is the same, and it is clear that at least one of the recursive branches 
produces a satisfying assignment. 

Hirsch’s algorithm is similar to the above sketch of the algorithm for the 
satisfiability promise problem, except that a different greedy strategy is used to 
pick the variables in V . The analysis is slightly different and somewhat more 
difficult. 



3 Pseudorandomness Against fc-CNF Formulae 

3.1 Some Technical Preliminaries 

We begin this section with a few technical definitions. 

We denote by C/„ the uniform distribution over {0, 1}”. If / : {0, 1}” ^ {0, 1} 
is a function and A is a distribution over {0, 1}", then we say that X e-fools f 
if 

|Pr[/(C/„) = l]-Pr[/(A) = I]| <£ 

If IF is a collection of functions, then we say that a distribution X £- fools T if 
X £-fools every function f G X. 

Our goal will be to find a distribution X that £-fools the class of fc-CNF 
formulae over n variables, and that is uniform over an efficiently constructable 
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support of polynomial size. Then, given a /c-CNF formula /, we can approximate 
Pr[/([/„) = 1] by computing the close value Pr[/(X) = 1], and we compute the 
latter by enumerating all the polynomially many elements in the support of X , 
and applying /() to each of them. We will show that £-biased distributions, 
defined below, can be used towards such goal. 

We say that a distribution X over {0, 1}" is £-biased [NN93] if for every 
subset S' C {1, . . . , n} we have 



- — £ < Pr 
2 



0Xi = 1 

-ieS 




Equivalently, we can say that a distribution is £-biased if it £-fools every linear 
function. (Where, of course, we mean linear over the field GF(2).) 

Theorem 1 ([NN93,AGHP92]). For every e, and n, there is an e-biased 
distribution over {0, 1}” that is uniform over a support of size polynomial in n 
and 1/e. Furthermore, the support can be constructed in time polynomial in n 
and 1/e. 

We say that a distribution X = {Xi • • ■ Xn) over {0, 1}”, where each Xi is 
unbiased, is k-wise independent if every k of the random variables X \,. . . , 
are mutually independent. Equivalently, a distribution is A:- wise independent if 
it 0-fools the class of functions that depend only on k or fewer input variables. 

We say that X is e-close to k-wise independent if for every function 
>{0,1} that depends on k or fewer inputs we have 



I P^[g{Un) = 1] - Pr[g(A) = i]| < £ 



that is, if X £-fools the class of functions that depend on at most k inputs. 

We say that X is e-close to k-wise independent in £oo norm if for every t < k, 
for every t indices i\, . . . ,it in (1, . . . , nj and for every t values ai, . . . ,at € (0, 1} 
we have 

— - £ < Pr[Ai^ = ai A ■ ■ ■ A Xi^ = at] < -^ -h e 

that is, if X £-fools the class of functions that can be expressed as checking that 
a subset of at most k bits of the input equals a particular sequence of values. 

The following connection between the notions that we have described is well 
known. 



Lemma 2. Let X be an e-biased distribution over {0,1}”. Then, for every k, 
X is £-2^/^-cZose to k-wise independent, and also 2e-close to k-wise independent 
in £oo norm. 



3.2 e-Biased Distribution and fc-CNF Fomulae 

In this section iwe give a proof of the following theorem. 
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Theorem 2 ([LV96]). There are functions t{k,e) = 0{k ■ 2^ • log(l/e)) and 
6{k,e) = such that the following happens. 

Let f : {0, 1}" ^ {0, 1} be a function defined by a k- CNF formula and let X 
be a distribution over {0, 1}" that is S{k,e)-close to t{k,e)-wise independent in 
£ao norm. Then X e-fools f . 

The application of Theorem 2 to e-biased distributions is immediate. 

Corollary 1. There is a function S'{k,e) = such that if X 

is a 6' {k,e) -biased distributions, then X e-fools every function computed by a 
k- CNF formula. 

The rest of this section is devoted to the proof of Theorem 2. 

Let / : {0, 1}” ^ {0, 1} be a function defined by a k-CNF formula (p over 
variables x\, . . . , Xn, and consider a decision tree over the variables x\, . . . , a;„. 
Every leaf of the decision tree (indeed, every node of the decision tree) defines 
a restriction, that is, an assignment to a subset of the variables x\,. . . , x„. If a 
leaf V is at distance t from the root, then it defines an assignment to t variables; 
if we pick a random assignment and then apply the decision tree to it, there is 
a probability 1 /2* that we reach the leaf v. In general, for a vertex v at distance 
t from the root we define the probability of v to be l/2‘, and for a set of vertices 
such that none of them is an ancestor of any other we define the probability of 
the set as the sum of the probabilities of the individual vertices. 

Lemma 3. Let f : {0, 1}” ^ {0, 1} be the function defined by a k-CNF formula 
(fi and e > 0. Let t be an integer such that (1 — 1/2^)* < e. Then there is 
a decision tree of depth at most tk such that: either (i) all the leaves define 
restrictions relative to which f is a constant, or (ii) all the leaves, except possibly 
a set of probability at most e, define restrictions relative to which (p becomes a 
{k-l)-CNF. 

Proof. We apply Lemma 1 to p with parameter t. Then we either find t clauses 
over disjoint variables or k{t — 1) variables that hit all the clauses. 

In the former case, consider the decision tree that reads all the < kt variables 
that occur in the t clauses. All but an e fraction of the leafs of the decision 
tree correspond to restrictions relative to which p is zero, and, in particular, is 
constant. 

In the latter case, consider the decision tree that reads all the < kt variables 
returned by the algorithm. All the leafes of the decision tree correspond to re- 
strictions relative to which is a (fc — 1)-CNF. □ 

We now compose the construction. 

Lemma 4. There is a function t{k,e) = 0(fc2^ ln(l/e)) such that for every 
f : {0, 1}" ^ {0, 1} defined by a k-CNF formula and for every £ > 0 there is 
a decision tree of depth at most t{k, s) such that all the leaves, except possibly 
a subset of probability e, correspond to a restriction relative to which f is a 
constant. 
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Proof. We prove the theorem by induction. Lemma 3 proves the theorem for 
k = 1 and t(l,£) = log 2 1/e. For general k, start by constructing the decision 
tree for / as in Lemma 3. If all but an e fraction of the leaves of the tree make / 
become a constant, then we are done. Otherwise, every leaf of the tree defines a 
restriction relative to which / is a (fc — 1)-CNF, and we can apply the induction 
hypothesis to get decision trees for each of these {k — 1)-CNF. 

This argument proves the theorem for every function t() that satisfies 
f(l,£) = log 2 1/e and t{k,e) > fc2^ ln(l/£)+t(/c— 1, e). In particular, the theorem 
is true for t(/c, £) = 2fc2^ ln(l/£). □ 

To prove Theorem 2 we now only need the following simple last step, which 
is well known. 

Lemma 5. Let f : {0, 1}” ^ {0, 1} be a function and T he a decision tree of 
depth t such that all but an e fraction of the leaves of T define a restriction 
relative to which f is a constant. Let X he a distribution that is 5-close to t-wise 
independent in loo norm. Then 

I Pr[/(C/„) = 1] - Pr[/(X) = 1]| < £ + <5 • 2‘ 

Proof. We may assume withouth loss of generality that T has 2* leaves, all at 
distance t from the root. (Otherwise, from leaves that are closer to the root, 
we read additional variables until we reach distance t. This does not change the 
properties of T assumed in the Lemma.) Let S be the set of leaves of T that 
define a restriction relative to which / is the constant 1. Then we have 

Pr[/(C/„) = 1]< ^ + £ 

If we sample an assignment according to AT, we see that for each leaf of T there 
is a probability at least 1/2* — 5 that the assignment is consistent with the leaf. 
In each of these event, / evaluates to one and, moreover, all these events are 
disjoint. We deduce 

Pr[f{X) = l]>\^-S-\Z\ 

and 

Pr[/(C/„) = 1] - Pr[/(X) = 1] < £ + <5 • 2‘ 

Similarly, we can prove 

Pr[f{X) = 1] - Pr[/(C/„) = 1] < £ + <5 • 2‘ 



□ 



4 Perspective 

The current body of work on derandomization (see [Kab02] for a survey) strongly 
suggests that every problem (including search problems and promise problems) 
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that is solvable probabilistically in polynomial time can also be solved deter- 
ministically in polynomial time. It is then a natural research program to look 
for deterministic polynomial time algorithms for all the interesting problems for 
which only probabilistic polynomial time algorithms are known. 

After the discovery of a deterministic polynomial time algorithm for test- 
ing primality [AKS02], the most interesting algorithms to derandomize are now 
the identity test for low-degree polynomials and the approximate counting al- 
gorithms based on the Markov-Chain Monte-Carlo approach. Kabanets and Im- 
pagliazzo [KI03] show that derandomizing the polynomial identity test algorithm 
for general arithmetic circuits implies the proof of circuit lower bounds that may 
be beyond our current proof techniques. It is not clear whether there are similar 
inherent difficulties in derandomizing approximate counting algorithms such as, 
say, the Permanent approximation algorithm of [JSVOl]. 

The problem of approximately counting the number of satisfying assignments 
for a given circuit up to a small additive error is clearly precisely the same 
problem as derandomizing every promise-BPP problem. (In particular, such an 
approximate counting algorithm would imply that NEXP ^ P/poly.) It seems 
possible, however, to derandomize at least bounded-depth circuits, and, at the 
very least, depth- two circuits in polynomial time using current techniques. In 
this paper we note that a special case of this problem can be solved by a simple 
divide-and-conquer algorithm, without using derandomization techniques. 
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